Unique ID: 2015103
| Division: | Office of Information and Regulatory Affairs |
|---|---|
| Issue Date: | February 13th 2019 |
| Last modified: | February 22nd 2019 |
Cost: FREE
Web Scrape and Application Program Interface (API): exploring webscraping on price data
Exploring webscraping on price data
Explore the possibility of scraping websites to obtain relevant data.
Project Objective:
Exploration, Pilot intended to go to production to supplement existing data
Project Outcomes:
Initial effort was used to supplement data to develop hedonic models. Additional efforts have been made to produce some exploratory price indices and compare to U.S. Bureau of Labor Statistics price indices
Statistical Area
Project Sources
| Type Of Institution: | National statistical office |
|---|---|
| Big Data Source: | Web scraping data |
| Region: | North America |
| Country Area: | United States |
| Id Country Regional: | country |
Partnerships
| Other Partners: | Other |
|---|
Accessing Data
| Data Access Rights: | Broader access rights |
|---|---|
| Intermediary: |
Data Coverage
| Data Coverage: | Other |
|---|---|
| Cost Implication: | Free |
| Cost Comments: | The net cost implication is undetermined. There is potential for reduction of collection costs, however development and maintenance costs exist. |
| Coverage Geo Comments: | Varies depending on the source. |
| Coverage Period: | Beginning in 2011 |
Project Details
| Frequency Comments: | Varies depending on the source. |
|---|
Data Quality
| Quality Framework: | Quality of output statistics |
|---|---|
| Quality Aspects Evaluated: | Privacy and Security, Completeness, Usability, Time Factors, Accessibility, Relevance, Validity, Accuracy, including selectivity |
| Validation Comments: | Validate using a variety of result comparisons and statistical techniques. |
| Data Quality Concerns Comments: | U.S. Bureau of Labor Statistics assesses the quality of all its data sources. |
| Data Quality Concerns: |
Methodology
| Methods Used: | Traditional statistical methods |
|---|---|
| Developed New Methods: |
Technologies
| Technologies: | Spreadsheet, Other |
|---|---|
| Technologies Comments: | Programming using Python and SAS |
Other
| Income Level: | High-income |
|---|---|
| Iso: | US |
| Timeframe To Produce Indicator: | NA |
| Frequency Comments: | Varies depending on the source. |
Write Your Own Review