Coursera 笔记

Module 04 · Career Opportunities and Data Engineering in Action职业机会与数据工程实践

Reading: Data Warehousing Specialist阅读:数据仓库专家

Coursera ↗

Objectives 目标

After completing this reading, you will be able to:
完成本阅读后,您将能够:

  • Describe the Data Warehousing Specialist role
    描述数据仓库专员的角色
  • Provide opportunity estimates for the Data Warehousing Specialist role
    提供数据仓库专员角色的机会估算
  • Indicate alternative job titles for the Data Warehousing Specialist role
    指出数据仓库专员角色的其他职位名称
  • List tasks performed, and skills required for the Data Warehousing Specialist role
    列出数据仓库专员角色的任务和所需技能
  • Describe career progression paths for the Data Warehousing Specialist role
    描述数据仓库专员的职业发展路径

Description of the Data Warehousing Specialist role 数据仓库专员角色描述

Data Warehousing Specialists design, model, and implement corporate data warehousing activities, program and configure warehouses of databases, and provide support to data warehouse users.
数据仓库专员负责设计、建模和实施企业数据仓库活动,对数据库仓库进行编程和配置,并为数据仓库用户提供支持。
The tasks for this specialist role are focussed on data warehousing and form a significant subset of the wider Data Engineering role.
此专业角色的任务侧重于数据仓库,是更广泛的数据工程角色中的一个重要子集。

Opportunity estimates for the Data Warehousing Specialist role 数据仓库专家角色的机会估算

According to careeronestop.org, an organization sponsored by the U.S. Department of Labour, the future is bright for Data Warehouse Specialists and very similar or related roles, such as Data Architects and Database Administrators. The expected growth rate of opportunities in these fields is higher than the average, and is expected to average about 8 to 10 percent per year over the next decade. Approximately 13,900 openings are expected to emerge each year in the U.S. alone.
根据美国劳工部赞助的组织 careeronestop.org 的数据,数据仓库专家以及数据架构师和数据库管理员等非常相似或相关的角色的未来前景光明。这些领域的机会预期增长率高于平均水平,预计未来十年每年平均增长约 8% 到 10%。仅在美国,预计每年将出现约 13,900 个职位空缺。

According to salary.com, the median salary for a Data Warehouse Specialist in the US is $110,168.
根据 salary.com 的数据,美国数据仓库专家的中位数年薪为 110,168 美元。

Data Warehousing Specialist alternative job titles 数据仓库专员的其他职位名称

Just like the role of Data Engineer, the Data Warehousing Specialist role is quite fluid and can vary considerably. In fact, the Data Warehousing Specialist is a particluar kind of Data Engineer, more tightly focused on the data warehousing aspects of the broader discipline. Accordingly, searching online job postings for this particular role returns many other closely related positions, including:
与数据工程师的角色一样,数据仓库专员的角色也相当灵活,并且可能差异很大。事实上,数据仓库专员是数据工程师的一种特殊类型,更专注于该广泛学科中与数据仓库相关的方面。因此,在线搜索此特定职位的招聘信息会返回许多其他密切相关的职位,包括:

  • Data Warehouse Specialist
    数据仓库专家
  • Data Warehouse Engineer  数据仓库工程师
  • Data Warehouse Solution Architect
    数据仓库解决方案架构师
  • Data Warehousing/ETL Solution Specialist
    数据仓库/ETL 解决方案专家
  • Data Warehouse Architect  数据仓库架构师
  • Data Architect, Data Warehousing & MPP
    数据架构师,数据仓库与 MPP
  • Data Warehouse Analyst  数据仓库分析师
  • Data Warehouse Administrator
    数据仓库管理员
  • Data Warehousing Development Specialist
    数据仓库开发专家

Tasks performed by Data Warehousing Specialists 数据仓库专家执行的任务

As a specialization within the broader field of Data Engineering, Data Warehousing Specialists may be responsible for many kinds of tasks. These tasks may include any of the following:
作为数据工程这一更广泛领域中的一个专业方向,数据仓库专家可能负责多种任务。这些任务可能包括以下任何一项:

  • Developing processes, procedures, and software applications for enterprise data management
    开发企业数据管理流程、程序和软件应用程序
  • Analyzing and improving data warehousing processes for efficiency, accuracy, usablilty, or security
    分析和改进数据仓库流程,以提高效率、准确性、可用性或安全性
  • Designing, modelling, or implementing corporate data warehousing activities
    设计、建模或实施企业数据仓库活动
  • Developing or maintaining standards, such as organization, structure, or nomenclature, for the design of data warehouse elements, such as data architectures, models, tools, and databases
    为数据仓库元素(如数据架构、模型、工具和数据库)的设计开发或维护标准,例如组织、结构或命名法
  • Providing or coordinating troubleshooting support for data warehouses
    为数据仓库提供或协调故障排除支持
  • Writing or modifying programs to meet customer requirements
    编写或修改程序以满足客户需求
  • Creating documentation such as metadata and diagrams of entity relationships, business processes, and process flow
    创建文档,例如元数据以及实体关系、业务流程和流程图的图表
  • Designing, implementing, or operating comprehensive data warehouse systems to balance optimization of data access with batch loading and resource utilization factors, according to customer requirements
    根据客户要求,设计、实施或操作综合数据仓库系统,以平衡数据访问优化与批量加载和资源利用因素
  • Performing system analysis, data analysis or programming, using a variety of computer languages and procedures
    使用各种计算机语言和程序进行系统分析、数据分析或编程
  • Reviewing designs, code, test plans, or documentation to ensure quality
    审查设计、代码、测试计划或文档以确保质量
  • Creating plans, test files, and scripts for data warehouse testing, ranging from unit to integration testing
    创建数据仓库测试的计划、测试文件和脚本,范围从单元测试到集成测试
  • Implementing business rules via stored procedures, middleware, or other technologies
    通过存储过程、中间件或其他技术实现业务规则
  • Supporting users of the data warehouse
    支持数据仓库用户

Soft skills required for the Data Warehousing Specialist role 数据仓库专员职位所需的软技能

  • Critical thinking - Using logic and reasoning to identify the strengths and weaknesses of alternative solutions, conclusions, or approaches to problems
    批判性思维——运用逻辑和推理来识别替代解决方案、结论或问题处理方法的优缺点
  • Complex problem solving - Identifying complex problems and reviewing related information to develop and evaluate options and implement solutions
    复杂问题解决——识别复杂问题并审查相关信息,以制定和评估方案并实施解决方案
  • Decision making - Assessing the costs and benefits of a variety of optional solutions, and choosing the most appropriate solution
    决策——评估各种可选解决方案的成本和收益,并选择最合适的解决方案
  • Systems evaluation - Identifying and evaluating system performance measures, and actions needed to improve or correct poor performance
    系统评估——识别和评估系统性能指标,以及改进或纠正不良性能所需的行动
  • Active learning - Understanding the implications of new information or technologies for both current and future problem-solving and decision-making
    主动学习——理解新信息或技术对当前和未来问题解决和决策的影响
  • Communicating effectively
    有效沟通

Technical Skills required for the Data Warehousing Specialist role 数据仓库专员所需的技术技能

Specialized knowledge  专业知识

  • Programming - writing computer programs for various purposes
    编程——为各种目的编写计算机程序
  • Systems analysis - determining how a business system works, and understanding how changes in conditions may affect outcomes
    系统分析——确定业务系统如何运作,并理解条件变化可能如何影响结果
  • Data architecture - understanding the models, the policies, rules or standards that govern which data is collected, how data is stored, arranged, and integrated, and how to put data to use in data systems
    数据架构——理解模型、策略、规则或标准,这些模型、策略、规则或标准管理着收集哪些数据、数据如何存储、排列和集成,以及如何在数据系统中利用数据
  • Data management and analysis - securely collecting, storing, and analyzing data
    数据管理和分析——安全地收集、存储和分析数据
  • Data pipelines - building and maintaining data pipelines
    数据管道——构建和维护数据管道
  • Business intelligence - data warehousing; extract, transform, and load (ETL); and data mapping
    商业智能——数据仓库;提取、转换和加载(ETL);以及数据映射
  • Database normalization - data integrity and normalization
    数据库规范化——数据完整性和规范化
  • Data storage structures - especially relational databases
    数据存储结构——特别是关系型数据库
  • Data migration - ETL of data from one system to another
    数据迁移——将数据从一个系统 ETL 到另一个系统
  • Metadata management and metadata standards
    元数据管理和元数据标准
  • Data integration platforms
    数据集成平台
  • Overview of the Data Engineering Ecosystem
    数据工程生态系统概述
  • Cloud data - building scalable cloud data infrastructure
    云数据——构建可扩展的云数据基础设施
  • Database administration, including Big Data administration
    数据库管理,包括大数据管理
  • Data acquisition and transformation - Digitizing data for display, analysis, and storage
    数据采集与转换——将数据数字化以进行显示、分析和存储
  • Data lakes, data marts, data reservoirs
    数据湖、数据集市、数据存储库
  • IoT - Integrating data from various connected devices and systems in IoT using data pipelines
    物联网——使用数据管道整合来自物联网中各种互联设备和系统的数据
  • Building event streaming pipelines
    构建事件流管道

Law & Government  法律与政府

  • Acts & Regulations - knowledge of laws, regulations, requirements and ethical issues related to the access and use of information, for example intellectual capital, personally identifiable information, and customer data
    法案与法规——了解与信息访问和使用相关的法律、法规、要求和道德问题,例如知识资本、个人身份信息和客户数据

Software and IT skills
软件和 IT 技能

  • Cloud computing and cloud platforms: Amazon Web Services (AWS), Microsoft Azure, SpringCloud, GCS (Google Cloud Storage)
    云计算和云平台:亚马逊云服务(AWS)、微软 Azure、SpringCloud、GCS(谷歌云存储)
  • Data warehouse tools: Snowflake, Data Bricks, BigQuery, Redshift, Db2
    数据仓库工具:Snowflake、Data Bricks、BigQuery、Redshift、Db2
  • Data pipeline tools: Apache Kafka, Apache Airflow, Luigi
    数据管道工具:Apache Kafka、Apache Airflow、Luigi
  • Big data tools: Apache Hadoop, Apache Spark, Apache Hive
    大数据工具:Apache Hadoop、Apache Spark、Apache Hive
  • Operating systems: UNIX, Linux
    操作系统:UNIX、Linux
  • Programming languages: SQL, Bash, Python, R, Java, C++
    编程语言:SQL、Bash、Python、R、Java、C++
  • Databases - Cassandra, Microsoft SQL Server, MySQL, PostgreSQL, Amazon DynamoDB, Apache Solr, IBM Db2, MongoDB, neo4j, Oracle PL/SQL
    数据库:Cassandra、Microsoft SQL Server、MySQL、PostgreSQL、Amazon DynamoDB、Apache Solr、IBM Db2、MongoDB、neo4j、Oracle PL/SQL
  • Metadata management software - CA Erwin Data Modeler; Oracle Warehouse Builder; SAS Data Integration Server; Talend Data Fabric; Alation Data Catalog, SAP Information Steward, Azure Data Catalog, IBM Watson Knowledge Catalog, Oracle Enterprise Metadata Management (OEMM), Adaptive Metadata Manager, Unifi Data Catalog, data.world, and Informatica Enterprise Data Catalog
    元数据管理软件:CA Erwin Data Modeler;Oracle Warehouse Builder;SAS Data Integration Server;Talend Data Fabric;Alation Data Catalog、SAP Information Steward、Azure Data Catalog、IBM Watson Knowledge Catalog、Oracle Enterprise Metadata Management (OEMM)、Adaptive Metadata Manager、Unifi Data Catalog、data.world 和 Informatica Enterprise Data Catalog
  • Agile software development methodologies
    敏捷软件开发方法
  • Version control - Git
    版本控制 - Git
  • Modelling and API development
    建模和 API 开发
  • Business intelligence and data analysis software - IBM Cognos Impromptu, MicroStrategy, Microsoft Power BI, Google Analytics, InsightSquared, Oracle Business Intelligence Enterprise Edition, Qlik Tech QlikView, ‎Sisense, ‎Tableau, ‎Dundas BI, ‎SAS Analytics, Domo, SAP Lumira
    商业智能和数据分析软件 - IBM Cognos Impromptu、MicroStrategy、Microsoft Power BI、Google Analytics、InsightSquared、Oracle Business Intelligence Enterprise Edition、Qlik Tech QlikView、Sisense、Tableau、Dundas BI、SAS Analytics、Domo、SAP Lumira

Pathways to becoming a Data Warehouse Specialist 成为数据仓库专家的途径

There are many possible paths to becoming a Data Warehousing Specialist. Most practitioners have a minimum of a Bachelor’s Degree in a mathematical or computational field such as Computer Science, Computer Engineering or the Mathematical Sciences. However, many practitioners instead have a Technology or Technical Diploma in a Computational or Information Technology discipline. Combining this educational background with some hands-on experience with application development and use of software for managing databases and metadata is a good way to prepare for the role.
成为数据仓库专家有许多可能的途径。大多数从业者至少拥有计算机科学、计算机工程或数学科学等数学或计算领域的学士学位。然而,许多从业者也拥有计算或信息技术学科的技术或专业文凭。将这种教育背景与应用程序开发以及数据库和元数据管理软件使用方面的实践经验相结合,是为担任此角色做好准备的好方法。

Career progression 职业发展

The career progression of a Data Warehouse Specialist might take a direct path, such as starting at the Junior or Associate Data Warehouse Specialist role, and evolving over time through the Data Warehouse Specialist, Senior Data Warehouse Specialist, Lead Data Warehouse Specialist, and Principal Data Warehouse Specialist roles. Having said that, the path of progression is by no means unique.
数据仓库专家的职业发展可能遵循一条直接的路径,例如从初级或助理数据仓库专家职位开始,并随着时间的推移逐步发展为数据仓库专家、高级数据仓库专家、首席数据仓库专家和首席数据仓库专家。话虽如此,职业发展路径绝非唯一。

Since the Data Warehouse Specialist role is simply a Data Engineering role with an emphasis on Data Warehousing, you will most likely find yourself shifting between specializations within the Data Engineering world, depending on your interests and abilities as well as the needs of the teams you find yourself working with.
由于数据仓库专员的角色只是一个侧重于数据仓库的数据工程角色,因此您很可能会根据自己的兴趣和能力以及所合作团队的需求,在数据工程领域内的专业方向之间进行转换。

On smaller teams you may need to participate day-to-day in many or all stages of the data engineering lifecycle. Working with larger teams, you will naturally find some combination of a smaller niche and career progression as the best fit. No matter your path, in order to progress throughout your career as an engineer, you will always need to be learning about and making use of emerging tools and technologies. This continuous learning will also inform you as you discover your path.
在规模较小的团队中,您可能需要日常参与数据工程生命周期的许多或所有阶段。在规模较大的团队中,您自然会发现更小的利基市场和职业发展相结合是最佳选择。无论您的道路如何,为了在工程师职业生涯中取得进步,您将始终需要学习和利用新兴工具和技术。这种持续学习也将为您发现自己的道路提供信息。

Summary 总结

In this reading, you learned that:
在本阅读中,您了解到:

  • The Data Warehousing Specialist is a particluar kind of Data Engineer, more tightly focused on the data warehousing aspects of the broader discipline
    数据仓库专员是一种特殊的数据工程师,更专注于该学科中与数据仓库相关的方面
  • Data Warehousing Specialists design, model, and implement corporate data warehousing activities, program and configure warehouses of databases, and provide support to data warehouse users
    数据仓库专员负责设计、建模和实施企业数据仓库活动,对数据库仓库进行编程和配置,并为数据仓库用户提供支持
  • Data Warehousing Specialists require specialized knowledge of data architecture - understanding the models, the policies, rules or standards that govern which data is collected, how data is stored, arranged, and integrated, and how to put data to use in data systems
    数据仓库专员需要具备数据架构方面的专业知识——理解模型、管理数据收集、存储、整理和集成方式以及如何在数据系统中使用数据的策略、规则或标准
  • Soft skills required for the data warehousing specialist role include complex problem solving and active learning
    数据仓库专员角色所需的软技能包括解决复杂问题和主动学习
  • No matter your path, in order to progress throughout your career as an engineer, you will always need to be learning about and making use of emerging tools and technologies
    无论您的职业道路如何,作为一名工程师,您若想在职业生涯中不断进步,就必须持续学习和利用新兴工具和技术