+ - 0:00:00
Notes for current slide
Notes for next slide

The Nature of Data

Understanding Human Experience, Experiments, and Where Data comes from


by Arvind Venkatadri

Written: July 13 2022

Updated: July 23 2022

1

What makes Human Experience?

How would we begin to describe this experience?

  • Where / When?
  • Who?
  • How?
  • How Big? How small? How frequent? How sudden?
  • And....How Surprising ! How Shocking! How sad...How Wonderful !!!
    So: Our Questions, and our Surprise lead us to creating Human Experiences.

https://www.anecdote.com/2014/09/story-framework/

2

Is This a Surprise?

3

The Element of Surprise?

Jane Austen knew a lot about human information processing as these snippets from Pride and Prejudice (published in 1813 -- over 200 years ago) show:

  • She was a woman of mean understanding, little information, and uncertain temper.
  • Catherine and Lydia had information for them of a different sort.
  • When this information was given, and they had all taken their seats, Mr. Collins was at leisure to look around him and admire,...
  • You could not have met with a person more capable of giving you certain information on that head than myself, for I have been connected with his family in a particular manner from my infancy.
  • This information made Elizabeth smile, as she thought of poor Miss Bingley.
  • This information, however, startled Mrs. Bennet ...

https://www.cs.bham.ac.uk/research/projects/cogaff/misc/austen-info.html

4

Human Experience is....Data??

6

Experiments and Hypotheses

A Kitchen Experiment

  • Inputs are: Ingredients, Recipes, Processes
  • Outputs are: Taste, Texture, Colour, Quantity!!
Used without permission from https://safetyculture.com/topics/design-of-experiments/
7

What is the Result of an Experiment?

All experiments give us data about phenomena

  • We obtain data about the things that happen: Outputs
  • What makes things happen?: Inputs
  • How?: Process
  • When? Factors
  • How much "output" is caused by how much "input"? Effect Size

All Experiments stem from Human Curiosity, a Hypothesis, and a Desire to Find out and Talk about Something

8

A Famous Lady and her Famous Experiment

In 1853, Turkey declared war on Russia. After the Russian Navy destroyed a Turkish squadron in the Black Sea, Great Britain and France joined with Turkey. In September of the following year, the British landed on the Crimean Peninsula and set out, with the French and Turks, to take the Russian naval base at Sevastopol.

What followed was a tragicomedy of errors -- failure of supply, failed communications, international rivalries. Conditions in the armies were terrible, and disease ate through their ranks. They finally did take Sevastopol a year later, after a ghastly assault. It was ugly business all around. Well over half a million soldiers lost their lives during the Crimean War.

9

Florence Nightingale's Data

Month Year Disease.rate Wounds.rate Other.rate
Apr 1854 1.4 0.0 7.0
May 1854 6.2 0.0 4.6
Jun 1854 4.7 0.0 2.5
Jul 1854 150.0 0.0 9.6
Aug 1854 328.5 0.4 11.9
Sep 1854 312.2 32.1 27.7
Oct 1854 197.0 51.7 50.1
Nov 1854 340.6 115.8 42.8
Dec 1854 631.5 41.7 48.0
Jan 1855 1022.8 30.7 120.0
10

How Does Data look Like, then?

Types of Variables:

Using Interrogative Pronouns

  • Nominal: What? Who? Where? (Factors, Dimensions)
  • Ordinal: Which Types? What Sizes? How Big? (Factors, Dimensions)
  • Interval: How Often? (Numbers, Facts)
  • Ratio: How many? How much? How heavy? (Numbers, Facts)

11

Types of Variables in Nightingale Data

Using Interrogative Pronouns:

  • Nominal: None
  • Ordinal: (Factors, Dimensions)
    • HOW? War, Disease, Other
  • Interval: (Numbers, Facts)
    • WHEN? Year, Month
  • Ratio: (Numbers, Facts)
    • HOW MANY? Rate of Deaths (War, Disease, Other)
Month Year Disease.rate Wounds.rate Other.rate
Apr 1854 1.4 0 7.0
May 1854 6.2 0 4.6
Jun 1854 4.7 0 2.5

Nightingale's data table had dimensions coded into column names. This is not considered tidy in the modern age

12

Nightingale's Rose

Nightingale created a remarkable and original graphical display to show us just what hadd really gone on in the War. It was a Polar-Area Diagram that showed how people had died during the period from July, 1854, through the end of the following year.

Nightingale's graph is like a pie chart, cut into twelve equal angles. These slices advance in a clockwise direction, one each month. The radius shows how many deaths occurred in that month. We see little short slices in April, May and June of 1854. After the troops land in the Crimea, the slices begin reaching far outward in the radial direction.

There's more: Each slice has three sections, one for deaths from wounds in battle, one for "other causes", and one for disease.

Once you see Nightingale's graph, the terrible picture is clear. The Russians were a minor enemy. The real enemies were cholera, typhus, and dysentery. Once the military looked at that eloquent graph, the modern army hospital system was inevitable.

13

So, Did the Sanitation Commission succeed?

14

Nightingale's famous Coxcomb or Rose Plot

"Engines of Our Ingenuity", https://www.uh.edu/engines/epi1712.htm

15

From Data -> Geometry

  • How did we arrive at shapes, colours, lines, points...from data?
  • All Statistical Graphs do a Kalidasa:
    • Transform a variable with a stat(count,bin,sort)
    • they use metaphors to map data variables and computed stats to geometrical aspects aka aesthetics

  • Commonly used aesthetics in data visualization: position, shape, size, color, line width, line type.
  • Some of these aesthetics can represent both continuous and discrete data (position, size, line width, color)
  • While others can usually only represent discrete data (shape, line type).
16

Each of the geometries works differently

17

The Need for Answers: Questions to Visuals

18

Variables and Graphs: Qualitative Variables

Amounts and Counts

  • Variable: Ordinal / Nominal
  • Stat: count
  • Geometry: height and colour
  • Questions:
    • How many of each type of #Var1?
    • How many of each type of #Var2 broken up by #Var2?

19

Variables and Graphs : Quantitative Variables

Distributions

  • Variable: Interval / Ratio
  • Stat: bin and count
  • Geometry: x = bins, y = count, and colour
  • Questions: Range and frequency of Interval/Ratio variable

20

Variables and Graphs : Quantitative Variable

Distributions (Many of them at once)

  • Variable: Interval/Ratio + Nominal/Ordinal
  • Stat: sort(boxplot), bin(violin)
  • Geometry: x = Nom/Ord, y = Int/Ratio, and colour = Nom/ord

21

Variables and Graphs : Quantitative Variables

X-Y Relationships

  • Variable: Interval/Ratio + Nominal/Ordinal
  • Stat: none
  • Geometry: x = Int/Ratio, y = Int/Ratio, and colour = Nom/ord

22

Conclusion

- We question the world and form Hypothesis out of surprise

- Hypotheses leads us to define Questions

- Questions lead to Variables

- Questions with Variables lead to Graphs

- With Graphs, we can write Stories ( Next Time!!)

23

Thanks!

Slides created via the R packages:

with

via the R packages:

⚔️ xaringan
+
😎 ✘gadenbuie/xaringanExtra
+
⚔️the tidyverse

30

What makes Human Experience?

How would we begin to describe this experience?

  • Where / When?
  • Who?
  • How?
  • How Big? How small? How frequent? How sudden?
  • And....How Surprising ! How Shocking! How sad...How Wonderful !!!
    So: Our Questions, and our Surprise lead us to creating Human Experiences.

https://www.anecdote.com/2014/09/story-framework/

2
Paused

Help

Keyboard shortcuts

, , Pg Up, k Go to previous slide
, , Pg Dn, Space, j Go to next slide
Home Go to first slide
End Go to last slide
Number + Return Go to specific slide
b / m / f Toggle blackout / mirrored / fullscreen mode
c Clone slideshow
p Toggle presenter mode
t Restart the presentation timer
?, h Toggle this help
oTile View: Overview of Slides
Alt + fFit Slides to Screen
sToggle scribble toolbox
Esc Back to slideshow