Coursera 笔记

Module 02 · Introduction to the Hadoop Ecosystem

Summary and Highlights: Introduction to Hadoop摘要和重点:Hadoop 简介

Coursera ↗

In this lesson, you learned that:
在本课中,你学习了:

  • Hadoop is an open-source framework for Big Data that faces challenges when encountering dependencies and low-level latency.
    Hadoop 是一个开源的大数据框架,在处理依赖关系和低级延迟时面临挑战。
  • MapReduce, a parallel computing framework used in parallel computing, is flexible for all data types, addresses parallel processing needs for multiple industries, and contains two major tasks, “map” and “reduce.”
    MapReduce 是一个用于并行计算的并行计算框架,它对所有数据类型都灵活,满足多个行业的并行处理需求,并包含两个主要任务:“map”和“reduce”。
  • The four main stages of the Hadoop Ecosystem are ingest, store, process and analyze, and access.
    Hadoop 生态系统的四个主要阶段是摄取、存储、处理和分析以及访问。
  • Key HDFS benefits include its cost efficiency, scalability, data storage expansion, and data replication capabilities. Rack awareness helps reduce the network traffic and improve cluster performance. HDFS enables “write once, read many” operations.
    HDFS 的主要优势包括其成本效益、可扩展性、数据存储扩展能力以及数据复制能力。"机架感知"有助于减少网络流量并提高集群性能。HDFS 支持"一次写入,多次读取"的操作。
  • Suited for static data analysis and built to handle petabytes of data, Hive is a data warehouse software for reading, writing, and managing datasets.
    Hive 适用于静态数据分析,并设计用于处理 PB 级数据,是一款用于读取、写入和管理数据集的数据仓库软件。
  • Hive is based on the “write once, read many” methodology, doesn’t enforce the schema to verify loading data, and has built-in partitioning support.
    Hive 基于"一次写入,多次读取"的方法论,不强制执行模式来验证加载数据,并具有内置的分区支持。
  • Linearly scalable and highly efficient, HBase is a column-oriented nonrelational database management system that runs on HDFS and provides an easy-to-use Java API for client access.
    HBase 具有线性可扩展性和高效率,是一款列式非关系型数据库管理系统,运行在 HDFS 上,并为客户端访问提供易于使用的 Java API。
  • The HBase architecture consists of HMaster, Region servers, Region, Zookeeper, and HDFS. A key difference between HDFS and HBase is that HBase allows dynamic changes compared to the rigid architecture of HDFS.
    HBase 的架构由 HMaster、Region 服务器、Region、Zookeeper 和 HDFS 组成。HDFS 与 HBase 之间一个关键的区别在于,与 HDFS 的刚性架构相比,HBase 允许动态变化。