Fun with data

[Resha Caner]

People tell me I am overly cynical and anti-science. Maybe they're right about the cynical part, but not the anti-science part. Rather, I see it as an attempt to maintain a high standard.

So, I thought it might be fun to share some of the things I've seen people do with data over my years as an engineer. Feel free to share your own. As a vehicle for the discussion, let's use a typical statistics problem: sampling marbles from an opaque bag. We have two possible types of marbles: steel and ceramic, and they are spheres of equal diameter.

1. I don't know how many marbles are in the bag, but I randomly sampled 3 and they were all steel. Therefore, the bag is filled with steel marbles.

2. There are 100,000 marbles in the bag. I sampled one and got steel. You questioned my result. I had enough money to sample two more times, and the result was 3 steel marbles. Therefore, I have significantly increased my confidence that the bag is filled with steel marbles.

3. My stat book says I need at least 30 samples. So, I got more money and took 30 random samples. Each time, the magnet drew out a steel marble. Therefore, the bag is filled with steel marbles.

4. I got rid of the magnet, and just draw with my hands. Since I can't see inside the bag, the best assumption is the marbles are randomly mixed. Therefore drawing 30 steel marbles means the bag is filled with steel marbles.

5. I now shake the bag to mix them before I make the draw of 30 marbles. We're good right?

6. OK. I've developed a process proven in a lab test to produce a random mixture. When I draw the marbles, I sort them by looking at them. Still all steel. We're good?

7. I've developed a measure of their properties. Steel is more dense, so when I draw the marbles, I weigh them. I need to make a correction because all the marbles are lighter than predicted, and therefore ceramic.

8. OK, I've developed some more tests. So, when I draw one marble I check it's color, chemistry, density, and magnetism. That means I've taken 4 samples for every marble drawn.

9. Oh, and I've now got 2 methods for drawing marbles. I alternate drawing the marbles with my left hand and then my right hand. Both methods produce the same result, so that increases my confidence.

10. Now that we know the first bag is all steel marbles, I can conclude this second bag is also all steel marbles.

11. We've developed a lab test that shows the marbles wear and leave dust behind. Steel marbles leave steel dust and ceramic marbles leave ceramic dust. I only found steel dust in the bag. So, even though I've not drawn any marbles, I know the bag only contains steel marbles.

Your turn.

 

[durangodawood]

11. could give you high confidence in all-steel depending on certain factors not elaborated upon in your example.

 

[AV1611VET]

Save your money.

Take one steel marble out of a bag.

Weigh it.

Multiply that weight by 100,000.

Any bag that doesn't have that exact weight contains ceramic marbles.*

* This is, of course, assuming all the marbles have the same density.

 

[OldWiseGuy]

* This is, of course, assuming all the marbles have the same density.

Don't you mean 'assuming that all the marbles (steel and ceramic) don't have the same density' (but are the same size)?

 

[Resha Caner]

Save your money.

Take one steel marble out of a bag.

Weigh it.

Multiply that weight by 100,000.

Any bag that doesn't have that exact weight contains ceramic marbles.*

* This is, of course, assuming all the marbles have the same density.

Look at #7. Hint: Weight does not determine density.

 

[Resha Caner]

11. could give you high confidence in all-steel depending on certain factors not elaborated upon in your example.

The examples are meant to be cumulative, and to give every action taken by the experimenter. So, in example #11 you don't know that there are any marbles in the bag at all, because note by this point we're actually talking about multiple bags of marbles.

No test can ever be complete. Practicality requires cutting it off somewhere. But establishing a level of confidence in the result is important. So, what factors would you add to #11 that, in your opinion, would give reasonable confidence?

 

[durangodawood]

The examples are meant to be cumulative, and to give every action taken by the experimenter. So, in example #11 you don't know that there are any marbles in the bag at all, because note by this point we're actually talking about multiple bags of marbles.

No test can ever be complete. Practicality requires cutting it off somewhere. But establishing a level of confidence in the result is important. So, what factors would you add to #11 that, in your opinion, would give reasonable confidence?

If you are stipulating "multiple bags of marbles", then you know there are marbles in each bag. Thats your 'given'.

I would be confident in zero ceramic marbles if they are highly likely to leave dust during their period in the bag, yet I find none in a competent search. Confidence level would track to that likelihood.

 

[bhsmte]

People tell me I am overly cynical and anti-science. Maybe they're right about the cynical part, but not the anti-science part. Rather, I see it as an attempt to maintain a high standard.

Do you feel you have a higher standard than science itself?

If so, what is it about your standard, that makes it better?

 

[Resha Caner]

If you are stipulating "multiple bags of marbles", then you know there are marbles in each bag. Thats your 'given'.

Ha! I'll give you that one. I should have just said "bags".

I would be confident in zero ceramic marbles if they are highly likely to leave dust during their period in the bag, yet I find none in a competent search. Confidence level would track to that likelihood.

OK.

 

[Resha Caner]

Do you feel you have a higher standard than science itself?

That's kind of an odd question. There is no single standard for science. The different professional organizations each set their own standard.

Regarding those disciplines in which I work, most of the time I accept the standards given me. But standards tend to be very generic and leave all kinds of room for error. It was more to that I was speaking.

With that said, I feel it's the obligation of all to push for improvement, and standards can always be improved. I could cite you two personal examples. In one case the standard made me feel morally uncomfortable. In the second case, the standard was woefully insufficient.

 

[bhsmte]

That's kind of an odd question. There is no single standard for science. The different professional organizations each set their own standard.

Regarding those disciplines in which I work, most of the time I accept the standards given me. But standards tend to be very generic and leave all kinds of room for error. It was more to that I was speaking.

With that said, I feel it's the obligation of all to push for improvement, and standards can always be improved. I could cite you two personal examples. In one case the standard made me feel morally uncomfortable. In the second case, the standard was woefully insufficient.

Why is it an odd question? You mentioned you tried to maintain a high standard and I am curious what you mean by that.

When dealing with humans, standards will indeed occasionally need to be improved.

Do you feel, that science in general, has the ability to recognize when standards need to improve?

 

[Resha Caner]

Do you feel, that science in general, has the ability to recognize when standards need to improve?

The question is odd because you keep using "science" as if it's a single living entity. I realize that's not what you mean, as you mentioned humans (plural) earlier in your post. Still, it makes it hard to understand what you're really asking.

For each person it's a learning process. It's not been my experience that the young engineers starting today use better standards than engineers from 25 years ago. They each have to start from ground zero and learn. So, it appears to me more a cycle than a steady progression. Errors are found and standards are added. But then the standards book becomes so large no one can absorb it all, and so people start ignoring them. That creates errors and the standards are refreshed.

It's not as if I'm the only one trying to improve standards. Again, that wasn't my purpose here. It was just meant as a bit of levity.

 

[bhsmte]

The question is odd because you keep using "science" as if it's a single living entity. I realize that's not what you mean, as you mentioned humans (plural) earlier in your post. Still, it makes it hard to understand what you're really asking.

Why is the question odd?

I am referring to professional scientists, who do various works in their field and scientists in general, who observe and peer review the work.

I was asking, whether the above, has the ability to recognize, when standards need to be upgraded.

 

[Resha Caner]

I was asking, whether the above, has the ability to recognize, when standards need to be upgraded.

It's not a yes or no question. As I said, the more experience someone has, the more likely they'll be able to recognize when standards need to be upgraded.

 

[bhsmte]

It's not a yes or no question. As I said, the more experience someone has, the more likely they'll be able to recognize when standards need to be upgraded.

You do believe there are professional scientists out there, that do recognize these things though, correct?

 

[Resha Caner]

You do believe there are professional scientists out there, that do recognize these things though, correct?

Of course.

 

[durangodawood]

....some of the things I've seen people do with data over my years as an engineer....

These people, are they schooled professionals? Average lay-people?

 

[Resha Caner]

These people, are they schooled professionals? Average lay-people?

Engineering colleagues.

 

[bhsmte]

Engineering colleagues.

And, it is not difficult for another professional to tell when data is not used properly?

 

[Resha Caner]

So, no one else in a science/technical profession has had some humorous encounters with data interpretation?