Course Text Book: 'Getting Started with Data Science' Publisher: IBM Press; 1 edition (Dec 13 2015) Print.
课程教材：《数据科学入门》 出版社：IBM 出版社；第 1 版（2015 年 12 月 13 日）印刷版。

Author: Murtaza Haider  作者：Murtaza Haider

Prescribed Reading: Chapter 12 Pg. 529-531
指定阅读：第 12 章 第 529-531 页

Establishing Data Mining Goals 建立数据挖掘目标

The first step in data mining requires you to set up goals for the exercise. Obviously, you must identify the key questions that need to be answered. However, going beyond identifying the key questions are the concerns about the costs and benefits of the exercise. Furthermore, you must determine, in advance, the expected level of accuracy and usefulness of the results obtained from data mining. If money were no object, you could throw as many funds as necessary to get the answers required. However, the cost-benefit trade-off is always instrumental in determining the goals and scope of the data mining exercise. The level of accuracy expected from the results also influences the costs. High levels of accuracy from data mining would cost more and vice versa. Furthermore, beyond a certain level of accuracy, you do not gain much from the exercise, given the diminishing returns. Thus, the cost-benefit trade-offs for the desired level of accuracy are important considerations for data mining goals.
数据挖掘的第一步要求您为该练习设定目标。显然，您必须确定需要回答的关键问题。然而，除了识别关键问题之外，还涉及该练习的成本和效益问题。此外，您必须事先确定数据挖掘所获得结果的预期准确性和实用性。如果金钱不是问题，您可以投入必要的资金来获取所需答案。然而，成本效益权衡始终在确定数据挖掘练习的目标和范围方面起着重要作用。对结果的预期准确度也会影响成本。数据挖掘的高准确度成本更高，反之亦然。此外，超过一定准确度水平后，由于边际效益递减，该练习不会带来太多收益。因此，对于所需准确度水平的成本效益权衡是数据挖掘目标的重要考虑因素。

Selecting Data 选择数据

The output of a data-mining exercise largely depends upon the quality of data being used. At times, data are readily available for further processing. For instance, retailers often possess large databases of customer purchases and demographics. On the other hand, data may not be readily available for data mining. In such cases, you must identify other sources of data or even plan new data collection initiatives, including surveys. The type of data, its size, and frequency of collection have a direct bearing on the cost of data mining exercise. Therefore, identifying the right kind of data needed for data mining that could answer the questions at reasonable costs is critical.
数据挖掘的输出在很大程度上取决于所使用数据的质量。有时，数据可以很容易地用于进一步处理。例如，零售商通常拥有大量关于客户购买和人口统计学的数据库。另一方面，数据可能不适合进行数据挖掘。在这种情况下，您必须识别其他数据来源，甚至计划新的数据收集计划，包括调查。数据的类型、大小和收集频率直接影响数据挖掘的成本。因此，确定所需的数据类型以进行数据挖掘，以合理成本回答问题至关重要。

Preprocessing Data 预处理数据

Preprocessing data is an important step in data mining. Often raw data are messy, containing erroneous or irrelevant data. In addition, even with relevant data, information is sometimes missing. In the preprocessing stage, you identify the irrelevant attributes of data and expunge such attributes from further consideration. At the same time, identifying the erroneous aspects of the data set and flagging them as such is necessary. For instance, human error might lead to inadvertent merging or incorrect parsing of information between columns. Data should be subject to checks to ensure integrity. Lastly, you must develop a formal method of dealing with missing data and determine whether the data are missing randomly or systematically.
数据预处理是数据挖掘的重要步骤。通常原始数据杂乱无章，包含错误或不相关的数据。此外，即使有相关数据，信息有时也会缺失。在预处理阶段，你需要识别数据中的无关属性，并将这些属性从进一步考虑中删除。同时，识别数据集的错误方面并将其标记为错误是必要的。例如，人为错误可能导致列间信息的无意合并或错误解析。数据应接受检查以确保完整性。最后，你必须制定一个处理缺失数据的正式方法，并确定数据是随机缺失还是系统性地缺失。

If the data were missing randomly, a simple set of solutions would suffice. However, when data are missing in a systematic way, you must determine the impact of missing data on the results. For instance, a particular subset of individuals in a large data set may have refused to disclose their income. Findings relying on an individual's income as input would exclude details of those individuals whose income was not reported. This would lead to systematic biases in the analysis. Therefore, you must consider in advance if observations or variables containing missing data be excluded from the entire analysis or parts of it.
如果数据是随机缺失的，一套简单的解决方案就足够了。然而，当数据以系统的方式缺失时，你必须确定缺失数据对结果的影响。例如，在一个大型数据集中，特定子集的人群可能拒绝透露他们的收入。依赖于个人收入作为输入的研究会排除那些未报告收入的人群的详细信息。这会导致分析中存在系统性偏差。因此，你必须提前考虑是否应将包含缺失数据的观察值或变量从整个分析或其部分中排除。

Transforming Data 数据转换

After the relevant attributes of data have been retained, the next step is to determine the appropriate format in which data must be stored. An important consideration in data mining is to reduce the number of attributes needed to explain the phenomena. This may require transforming data. Data reduction algorithms, such as Principal Component Analysis (demonstrated and explained later in the chapter), can reduce the number of attributes without a significant loss in information. In addition, variables may need to be transformed to help explain the phenomenon being studied. For instance, an individual's income may be recorded in the data set as wage income; income from other sources, such as rental properties; support payments from the government, and the like. Aggregating income from all sources will develop a representative indicator for the individual income.
在保留数据的相关属性后，下一步是确定数据存储的适当格式。数据挖掘中的一个重要考虑是减少解释现象所需的属性数量。这可能需要转换数据。数据缩减算法，如主成分分析（本章后面将演示和解释），可以在不显著损失信息的情况下减少属性数量。此外，变量可能需要转换以帮助解释所研究的现象。例如，个人收入可能在数据集中记录为工资收入；来自其他来源的收入，如租金收入；来自政府的补助金等。汇总所有来源的收入将形成一个代表个人收入的指标。

Often you need to transform variables from one type to another. It may be prudent to transform the continuous variable for income into a categorical variable where each record in the database is identified as low, medium, and high-income individual. This could help capture the non-linearities in the underlying behaviors.
通常你需要将变量从一种类型转换为另一种类型。将收入的连续变量转换为分类变量可能是明智的，其中数据库中的每条记录都被识别为低收入、中等收入和高收入个人。这有助于捕捉潜在行为中的非线性特征。

Storing Data 数据存储

The transformed data must be stored in a format that makes it conducive for data mining. The data must be stored in a format that gives unrestricted and immediate read/write privileges to the data scientist. During data mining, new variables are created, which are written back to the original database, which is why the data storage scheme should facilitate efficiently reading from and writing to the database. It is also important to store data on servers or storage media that keeps the data secure and also prevents the data mining algorithm from unnecessarily searching for pieces of data scattered on different servers or storage media. Data safety and privacy should be a prime concern for storing data.
转换后的数据必须以有利于数据挖掘的格式进行存储。数据必须以赋予数据科学家无限制和即时读写权限的格式进行存储。在数据挖掘过程中，会创建新的变量，这些变量会写回原始数据库，因此数据存储方案应便于高效地从数据库中读取和写入。同时，将数据存储在能够保持数据安全并防止数据挖掘算法不必要地搜索分散在不同服务器或存储介质上的数据片段的服务器或存储介质上也很重要。数据安全和隐私应该是存储数据的首要考虑因素。

Mining Data 数据挖掘

After data is appropriately processed, transformed, and stored, it is subject to data mining. This step covers data analysis methods, including parametric and non-parametric methods, and machine-learning algorithms. A good starting point for data mining is data visualization. Multidimensional views of the data using the advanced graphing capabilities of data mining software are very helpful in developing a preliminary understanding of the trends hidden in the data set.
数据经过适当处理、转换和存储后，即可进行数据挖掘。这一步骤涵盖数据分析方法，包括参数方法和非参数方法，以及机器学习算法。数据挖掘的良好起点是数据可视化。利用数据挖掘软件的高级绘图功能对数据进行多维视图展示，对于初步理解数据集中隐藏的趋势非常有帮助。

Later sections in this chapter detail data mining algorithms and methods.
本章后续部分将详细阐述数据挖掘算法和方法。

Evaluating Mining Results 评估挖掘结果

After results have been extracted from data mining, you do a formal evaluation of the results. Formal evaluation could include testing the predictive capabilities of the models on observed data to see how effective and efficient the algorithms have been in reproducing data. This is known as an "in-sample forecast". In addition, the results are shared with the key stakeholders for feedback, which is then incorporated in the later iterations of data mining to improve the process.
从数据挖掘中提取结果后，需要对结果进行正式评估。正式评估可能包括在观测数据上测试模型的预测能力，以查看算法在再现数据方面的有效性和效率。这被称为“样本内预测”。此外，结果将与主要利益相关者共享以获取反馈，这些反馈将在数据挖掘的后续迭代中融入，以改进流程。

Data mining and evaluating the results becomes an iterative process such that the analysts use better and improved algorithms to improve the quality of results generated in light of the feedback received from the key stakeholders.
数据挖掘和评估结果成为一个迭代过程，分析师利用更优化的算法来改进结果质量，以根据关键利益相关者提供的反馈进行改进。