Challenges for Official Statistics

• Academic economist at Cornell University
• Occassional creator of Official Statistics at the Census Bureau (QWI and National QWI [2])
• NSF-Census Research Network grant holder and coordinator

*

Provide

• high-quality
• cost-effective statistics to
• inform decisions made by a
• wide range of stakeholders

there are many:

• budget cuts
• increasing privacy concerns
• challenge from alternate data sources
• increased expectations in an era of information
• challenge to assess new stakeholder needs (who are the clients?)
• changes in institutional culture (imposed and expected)

• budget cuts

• challenge from alternate data sources: collection, multiplicity
• increased privacy concerns and stakeholder needs
• computational challenges
• changes in institutional culture

• 8 GB? I can't run it on my laptop, my RA might
• 2 GB/day? 7.5TB? 2 years worth of stock trades
• 309.3 million people, measured once? 10 questions on US decennial census
• 150 million people, measured 80 times? LEHD US linked employer-employee data
• 16,721,787,543 tables? QWI derived from LEHD)
• 100 countries, 5-15 times? *IPUMS
• 19.3 billion records reduced to 112 weekly data points? = 10% of tweets over 24 months, UM Economic Indicators from Social Media (now up to twice that)

• surveys:
  • well established science,
  • trained professionals designing, fielding, analyzing surveys
• administrative data:
  • aim is to administer programs
  • definition of population, frame critical, but ex-post
  • statistics about populations is secondary purpose
• organic data
  • aim is … something else (Twitter?)
  • population and frame often unclear

• defining population
• estimating variability measures
• computing statistics

• treating admin/organic as a noisy data source, different from surveys
• designing administrative data collection with statistics in mind
• handling large data flows in commonly accepted ways
• novel confidentiality issues when ingesting other data sources
• reconceptualizing multiciplity of data sources

“Respondent load should always be considered when planning a statistical collection and there should be policies and practices in place to manage relationships with respondents. The aim should always be to keep reporting load to the minimum and to maintain the high quality of collections.”

Australian National Statistical Service

• ACS-based county-to-county flows (5-year horizon) [survey]
• Tax-filing-based county-to-county flows (Yearly horizon) [admin: IRS]
• Workplace to residence flows (Daily horizon?) [matched admin]
• Workplace based county-to-county (job) flows (Quarterly horizon published, others possible) [admin: states]

• Underlying population is the same
• Discrete data products
• Reconciling differences

What about using organic data?

Travel tweets

Just Landed - 36 Hours by blprnt

• Representativeness?
• Reconciling differences is a challenge

• no competition, right?

… they all refer back to official statistics!

• Timeliness
• Data collection by non-NSO

• More detailed data implies more data on specific individuals and firms
• Are protection methods sufficient?

• apply methods, produce more public-use statistics (move the data to the researcher)
• fail to apply methods, provide controlled direct access to data (move the researcher to the data; contracts/RDCs/etc.)

• older methods break down as published data become denser
• newer methods are still being developed
  • synthetic data
  • noise infusion
• more robust methods
  • differential privacy

• tradeoff utility - protection
• how much protection do data providers require (people, firms, etc.)
• how much utility do stakeholders request (people, firms, government, etc.)
• what technology is available to implement?

Abowd and Schmutte, Revisiting the Economics of Privacy: Population Statistics and Privacy as Public Goods

• public use statistics as a form of efficient distribution (aggregation)
• also allows for distributed computing!

Allow researchers to explore the entire multi-dimensional distribution, including its extremes, for instance for rare events or measures of inequality or program impact.

• Large Hadron Collider (since 2008)
• Arecibo Radiotelescope (since 1963)

Computing gap

• Census Bureau vs. XSEDE
• Statistics Canada vs. Compute Canada

Challenges in

• moving data to compute resource, or
• compute resource to data

Administrative data and organic data increase the challenge

• computing approaches require a change in procedures
• openness to new research methods (not in statistics!) and new skill sets (not in statistics!)
• academic-agency collaborations are an important part

“… the current Census Bureau survey and census methods are unsustainable. Changes must occur in the acquisition of data and construction of statistical information for the Census Bureau to succeed.”

Robert Groves, Director, Census Bureau, September 8, 2011

“Modern computational tools play the same role now that survey design and implementation did in the 1960s.”

John Abowd and Steve Fienberg, CNSTAT, May 8, 2015

“We would like to suggest […] implementing a variety of new models for facilitating the movements of researchers between academia and the Federal Statistical System”

Report of the NSF reverse site visit of the NCRN, April 2015

• challenge from alternate data sources:
  • open up to alternate data sources
  • new science on how to address distributed, independently generated data sources
• increased privacy concerns and stakeholder needs
  • investigate new methods to address privacy issues (access, disclosure avoidance techniques)
  • better toolkit to identify the right balance between stakeholder utility and data provider privacy
• computational challenges
  • shift in computational infrastructure (both production and research; also access)
• changes in institutional culture
  • increased focus on interactions with academia
  • expansion of staff skill diversity, in addition to traditional skill sets