Unique ID: 2015103

Division: Office of Information and Regulatory Affairs
Issue Date: February 13th 2019
Last modified: February 22nd 2019
Collaborative

Web Scrape and Application Program Interface (API): exploring webscraping on price data

Exploring webscraping on price data

Explore the possibility of scraping websites to obtain relevant data.

Project Objective:

Exploration, Pilot intended to go to production to supplement existing data

Project Outcomes:

Initial effort was used to supplement data to develop hedonic models. Additional efforts have been made to produce some exploratory price indices and compare to U.S. Bureau of Labor Statistics price indices

Project Sources
Project Sources
Type Of Institution: National statistical office
Big Data Source: Web scraping data
Region: North America
Country Area: United States
Id Country Regional: country
Partnerships
Partnerships
Other Partners: Other
Accessing Data
Accessing Data
Data Access Rights: Broader access rights
Data Coverage
Data Coverage
Data Coverage: Other
Cost Implication: Free
Cost Comments: The net cost implication is undetermined. There is potential for reduction of collection costs, however development and maintenance costs exist.
Coverage Geo Comments: Varies depending on the source.
Coverage Period: Beginning in 2011
Project Details
Project Details
Frequency Comments: Varies depending on the source.
Data Quality
Data Quality
Quality Framework: Quality of output statistics
Quality Aspects Evaluated: Privacy and Security, Completeness, Usability, Time Factors, Accessibility, Relevance, Validity, Accuracy, including selectivity
Validation Comments: Validate using a variety of result comparisons and statistical techniques.
Data Quality Concerns Comments: U.S. Bureau of Labor Statistics assesses the quality of all its data sources.
Methodology
Methodology
Methods Used: Traditional statistical methods
Technologies
Technologies
Technologies: Spreadsheet, Other
Technologies Comments: Programming using Python and SAS
Other
Other
Income Level: High-income
Iso: US
Timeframe To Produce Indicator: NA
Frequency Comments: Varies depending on the source.
Write Your Own Review
You're reviewing:Web Scrape and Application Program Interface (API): exploring webscraping on price data
Your Rating