Coursera 笔记

Module 01 · What is Data Engineering?什么是数据工程?

Summary and Highlights总结与要点

Coursera ↗

In this lesson, you have learned:
在本课程中,您学习了:

The role of a Data Engineer includes:
数据工程师的角色包括:

  • Gathering data from disparate sources.
    从不同的来源收集数据。
  • Integrating data into a unified view for data consumers.
    将数据整合为一个统一的视图,供数据消费者使用。
  • Preparing data for analytics and reporting.
    为分析和报告准备数据。
  • Managing data pipelines for a continuous flow of data from source to destination systems.
    管理数据管道,以实现从源系统到目标系统的数据持续流动。
  • Managing the complete infrastructure for the collection, processing, and storage of data.
    管理数据收集、处理和存储的完整基础设施。

To be successful in their role, Data Engineers need a mix of technical, functional, and soft skills.
要成功履行其职责,数据工程师需要具备技术、职能和软技能的混合能力。

  • Technical Skills include working with different operating systems and infrastructure components such as virtual machines, networks, and application services. It also includes working with databases and data warehouses, data pipelines, ETL tools, big data processing tools, and languages for querying, manipulating, and processing data.
    技术技能包括使用不同的操作系统和基础设施组件,如虚拟机、网络和应用服务。它还包括使用数据库和数据仓库、数据管道、ETL 工具、大数据处理工具以及用于查询、操作和处理数据的语言。
  • An understanding of the potential application of data in business is an important skill for a data engineer. Other functional skills include the ability to convert business requirements into technical specifications, an understanding of the software development lifecycle, and the areas of data quality, privacy, security, and governance.
    理解数据在业务中的潜在应用是数据工程师的重要技能。其他职能技能包括将业务需求转化为技术规范的能力、对软件开发生命周期的理解,以及数据质量、隐私、安全和治理领域。
  • Soft Skills include interpersonal skills, the ability to work collaboratively, teamwork, and effective communication.
    软技能包括人际交往能力、协作能力、团队合作和有效沟通。