Module 02 · Final Project期末项目
Practice Project Overview练习项目概述
Estimated Effort : 5 mins 预计用时:5 分钟
In this practice project, you will use the skills acquired through the course and create a complete ETL pipeline for accessing data from a website and processing it to meet the requirements.
在此实践项目中,您将运用在课程中获得的技能,创建一个完整的 ETL 管道,用于从网站访问数据并对其进行处理以满足要求。
Project Scenario 项目场景
An international firm that is looking to expand its business in different countries across the world has recruited you. You have been hired as a junior Data Engineer and are tasked with creating an automated script that can extract the list of all countries in order of their GDPs in billion USDs (rounded to 2 decimal places), as logged by the International Monetary Fund (IMF). Since IMF releases this evaluation twice a year, this code will be used by the organization to extract the information as it is updated.
一家希望在全球不同国家拓展业务的国际公司聘请了您。您被聘为初级数据工程师,任务是创建一个自动化脚本,该脚本能够提取国际货币基金组织(IMF)记录的所有国家按其 GDP(以十亿美元计,四舍五入到小数点后两位)排序的列表。由于 IMF 每年发布两次此评估,因此该组织将使用此代码在信息更新时提取信息。
You can find the required data on this webpage.
您可以在此网页上找到所需数据。
The required information needs to be made accessible as a JSON file 'Countries_by_GDP.json' as well as a table 'Countries_by_GDP' in a database file 'World_Economies.db' with attributes 'Country' and 'GDP_USD_billion.'
所需信息需要以 JSON 文件“Countries_by_GDP.json”以及数据库文件“World_Economies.db”中的表“Countries_by_GDP”的形式提供,其中包含属性“Country”和“GDP_USD_billion”。
Your boss wants you to demonstrate the success of this code by running a query on the database table to display only the entries with more than a 100 billion USD economy. Also, log the entire process of execution in a file named 'etl_project_log.txt'.
您的老板希望您通过对数据库表运行查询来展示此代码的成功,以仅显示经济规模超过 1000 亿美元的条目。此外,将整个执行过程记录在一个名为“etl_project_log.txt”的文件中。
You must create a Python code etl_project_gdp.py that performs all the required tasks.
您必须创建一个 Python 代码 etl_project_gdp.py 来执行所有必需的任务。