Documentation
Related Projects
Welcome to Apache& Hadoop&!
What Is Apache Hadoop?
The Apache& Hadoop& project develops open-source software for
reliable, scalable, distributed computing.
The Apache Hadoop software library is a framework that
allows for the distributed processing of large data sets across clusters
of computers using simple programming models. It is designed to scale up
from single servers to thousands of machines, each offering local
computation and storage. Rather than rely on hardware to deliver
high-availability, the library itself is designed to detect and handle
failures at the application layer, so delivering a highly-available
service on top of a cluster of computers, each of which may be prone to
The project includes these modules:
Hadoop Common:
The common utilities that support the other Hadoop modules.
Hadoop Distributed File System (HDFS&):
A distributed file system that provides high-throughput access to application data.
Hadoop YARN:
A framework for job scheduling and cluster resource management.
Hadoop MapReduce:
A YARN-based system for parallel processing of large data sets.
Other Hadoop-related projects at Apache include:
A web-based tool for provisioning, managing, and monitoring Apache Hadoop
clusters which includes support for Hadoop HDFS, Hadoop MapReduce, Hive,
HCatalog, HBase, ZooKeeper, Oozie, Pig and Sqoop. Ambari also provides
a dashboard for viewing cluster health such as heatmaps and ability to
view MapReduce, Pig and Hive applications visually alongwith features to
diagnose their performance characteristics in a user-friendly manner.
A data serialization system.
A scalable multi-master database with no single points of failure.
A data collection system for managing large distributed systems.
A scalable, distributed database that supports structured data storage for large tables.
A data warehouse infrastructure that provides data summarization and ad hoc querying.
A Scalable machine learning and data mining library.
A high-level data-flow language and execution framework for parallel computation.
A fast and general compute engine for Hadoop data. Spark provides a simple and
expressive programming model that supports a wide range of applications,
including ETL, machine learning, stream processing, and graph computation.
A generalized data-flow programming framework, built on Hadoop YARN,
which provides a powerful and flexible engine to execute an arbitrary
DAG of tasks to process data for both batch and interactive use-cases.
Tez is being adopted by Hive&, Pig& and other frameworks in
the Hadoop ecosystem, and also by other commercial software (e.g. ETL tools),
to replace Hadoop& MapReduce as the underlying execution engine.
A high-performance coordination service for distributed applications.
Getting Started
To get started, begin here:
reading the documentation.
Hadoop from the
release page.
Hadoop on the
mailing list.
Download Hadoop
Please head to the
page to download a release of Apache Hadoop.
Who Uses Hadoop?
A wide variety of companies and organizations use Hadoop for both research and production.
Users are encouraged to add themselves to the Hadoop
wiki page.
11 February, 2016: Release 2.6.4 available
A point release for the 2.6 line.
Please see the
for the list of 46 critical bug
fixes and since the previous release 2.6.3.
25 January, 2016: Release 2.7.2 (stable) available
A point release for the 2.7 line.
Please see the
for the list of 155 bug fixes and
patches since the previous release 2.7.1.
17 December, 2015: Release 2.6.3 available
A point release for the 2.6 line.
Please see the
for the list of 35 critical bug
fixes and since the previous release 2.6.2.
28 October, 2015: Release 2.6.2 available
A point release for the 2.6 line.
Please see the
for the list of 15 critical bug
fixes and since the previous release 2.6.1.
23 September, 2015: Release 2.6.1 available
A point release for the 2.6 line.
Please see the
for the list of 158 critical bug
fixes and since the previous release 2.6.0.
06 July, 2015: Release 2.7.1 (stable) available
A point release for the 2.7 line. This release is now considered
Please see the
for the list of 131 bug fixes and
patches since the previous release 2.7.0. Please look at the
2.7.0 section below for the list of enhancements enabled by this
first stable release of 2.7.x.
21 April 2015: Release 2.7.0 available
Apache Hadoop 2.7.0 contains a number of significant
enhancements. A few of them are noted below.
IMPORTANT notes
This release drops support for JDK6 runtime and works with
JDK 7+ only.
This release is not yet ready for production use. Critical
issues are
being ironed out via testing and downstream
adoption. Production users should wait for a 2.7.1/2.7.2
Hadoop Common
Support Windows Azure Storage - Blob as a file
system in Hadoop.
Hadoop HDFS
Support for file truncate
Support for quotas per storage type
Support for files with variable-length blocks
Hadoop YARN
Make YARN authorization pluggable
Automatic shared, global caching of YARN localized resources
Hadoop MapReduce
Ability to limit running Map/Reduce tasks of a job
Speed up FileOutputCommitter for very large jobs with many
output files.
Full information about this milestone release is available at
18 November, 2014: release 2.6.0 available
Apache Hadoop 2.6.0 contains a number of significant
enhancements such as:
Hadoop Common
Key management server (beta)
Credential provider (beta)
Hadoop HDFS
Heterogeneous Storage Tiers - Phase 2
Application APIs for heterogeneous storage
SSD storage tier
Memory as a storage tier (beta)
Support for Archival Storage
Transparent data at rest encryption (beta)
Operating secure DataNode without requiring root
Hot swap drive: support add/remove data node volumes
without restarting data node (beta)
AES support for faster wire encryption
Hadoop YARN
Support for long running services in YARN
Service Registry for applications
Support for rolling upgrades
Work-preserving restarts of ResourceManager
Container-preserving restart of NodeManager
Support node labels during scheduling
Support for time-based resource reservations in
Capacity Scheduler (beta)
Global, shared cache for application artifacts (beta)
Support running of applications natively in
Docker containers (alpha)
Full information about this milestone release is available at
19 November, 2014: release 2.5.2 available
Full information about this milestone release is available at
12 September, 2014: release 2.5.1 available
Full information about this milestone release is available at
11 August, 2014: release 2.5.0 available
Full information about this milestone release is available at
30 June, 2014: release 2.4.1 available
Full information about this milestone release is available at
27 June, 2014: release 0.23.11 available
Full information about this milestone release is available at
07 April, 2014: release 2.4.0 available
Full information about this milestone release is available at
20 February, 2014: release 2.3.0 available
Full information about this milestone release is available at
11 December, 2013: release 0.23.10 available
Full information about this milestone release is available at
15 October, 2013: release 2.2.0 available
Apache Hadoop 2.x reaches GA milestone!
Full information about this milestone release is available at
25 August, 2013: release 2.1.0-beta available
Apache Hadoop 2.x reaches beta milestone!
Full information about this milestone release is available at
27 December, 2011: release 1.0.0 available
Hadoop reaches 1.0.0!
Full information about this milestone release is available at
March 2011 - Apache Hadoop takes top prize at Media Guardian Innovation Awards
Described by the judging panel as a "Swiss army knife of the 21st century",
Apache Hadoop picked up the innovator of the year award for having the
potential to change the face of media innovations.
January 2011 - ZooKeeper Graduates
Hadoop's ZooKeeper subproject has graduated to become a
top-level Apache project.
Apache ZooKeeper can now be found
September 2010 - Hive and Pig Graduate
Hadoop's Hive and Pig subprojects have graduated to become
top-level Apache projects.
Apache Hive can now be found
Pig can now be found
May 2010 - Avro and HBase Graduate
Hadoop's Avro and HBase subprojects have graduated to become
top-level Apache projects.
Apache Avro can now be found
Apache HBase can now be found
July 2009 - New Hadoop Subprojects
Hadoop is getting bigger!
Hadoop Core is renamed Hadoop Common.
MapReduce and the Hadoop Distributed File System (HDFS) are now separate subprojects.
Avro and Chukwa are new Hadoop subprojects.
See the summary descriptions for all subprojects above. Visit the individual sites for more detailed information.
March 2009 - ApacheCon EU
In case you missed it....
November 2008 - ApacheCon US
In case you missed it....
July 2008 - Hadoop Wins Terabyte Sort Benchmark
One of Yahoo's Hadoop clusters sorted 1 terabyte of data in 209 seconds, which beat the previous record of 297 seconds in the annual general purpose
(Daytona) . This is the first time that either a Java or an open source program has won.查看: 13094|回复: 3
开发hadoop2.5程序，需要导入那些jar包？
主题帖子积分
注册会员, 积分 93, 距离下一级还需 107 积分
注册会员, 积分 93, 距离下一级还需 107 积分
开发hadoop2.5程序，需要导入那些jar包？
主题帖子积分
注册会员, 积分 93, 距离下一级还需 107 积分
注册会员, 积分 93, 距离下一级还需 107 积分
自己解决了：
hadoop-2.5.0\share\hadoop\common&&所有jar，
hadoop-2.5.0\share\hadoop\common\lib&&所有jar，
hadoop-2.5.0\share\hadoop\hdfs&&所有jar
hadoop-2.5.0\share\hadoop\mapreduce&&所有jar
hadoop-2.5.0\share\hadoop\yarn&&所有jar
能力很强悍&
主题帖子积分
高级会员, 积分 1980, 距离下一级还需 3020 积分
高级会员, 积分 1980, 距离下一级还需 3020 积分
这里大部分都没有细分是哪个包，我们在开发过程中，导入包的目的主要是解决语法问题，也就是说不让自己的程序在编写阶段就报错。
但是我们完全可以不导入包，采用文本的开发方式，写完程序之后，我们直接放到集群上，就可以跑程序了。
又由于hadoop的包，还是比较多的，而且存在比较多的依赖，如果每次都编程都思考我们需要哪些包的话，恐怕，没进行一步，都需要导入一些包。
所以为了避免麻烦，我们可以把hadoop的包都导入。
导包的方式可以参考，尽管这个是hadoop1.但是在hadoop2中，如果有Java基础，相信找到包，不是难事
欢迎加入about云群 、 ，云计算爱好者群，关注
积极上进，爱好学习
经常参与各类话题的讨论，发帖内容较有主见
经常帮助其他会员答疑
站长推荐 /6
about云|新出视频，openstack零基础入门，解决你ping不通外网难题
云计算hadoop视频大全(新增 yarn、flume|storm、hadoop一套视频
视频资料大优惠
大数据零基础由入门到实战
阶段1：hadoop零基础入门基础篇
阶段2：hadoop2入门
阶段3：大数据非hadoop系列课程
阶段4：项目实战篇
阶段5：大数据高级系列应用课程
阶段6：工作实用系列教程
等待验证会员请验证邮箱
新手获取积分方法
Powered by二次元同好交流新大陆
扫码下载App
汇聚2000万达人的兴趣社区下载即送20张免费照片冲印
扫码下载App
温馨提示！由于新浪微博认证机制调整，您的新浪微博帐号绑定已过期，请重新绑定！&&|&&
LOFTER精选
网易考拉推荐
用微信&&“扫一扫”
将文章分享到朋友圈。
用易信&&“扫一扫”
将文章分享到朋友圈。
阅读(486)|
用微信&&“扫一扫”
将文章分享到朋友圈。
用易信&&“扫一扫”
将文章分享到朋友圈。
历史上的今天
在LOFTER的更多文章
loftPermalink:'',
id:'fks_',
blogTitle:'hadoop2.2安装pig0.12问题',
blogAbstract:'
{if x.moveFrom=='wap'}
{elseif x.moveFrom=='iphone'}
{elseif x.moveFrom=='android'}
{elseif x.moveFrom=='mobile'}
${a.selfIntro|escape}{if great260}${suplement}{/if}
{list a as x}
推荐过这篇日志的人：
{list a as x}
{if !!b&&b.length>0}
他们还推荐了：
{list b as y}
转载记录：
{list d as x}
{list a as x}
{list a as x}
{list a as x}
{list a as x}
{if x_index>4}{break}{/if}
${fn2(x.publishTime,'yyyy-MM-dd HH:mm:ss')}
{list a as x}
{if !!(blogDetail.preBlogPermalink)}
{if !!(blogDetail.nextBlogPermalink)}
{list a as x}
{if defined('newslist')&&newslist.length>0}
{list newslist as x}
{if x_index>7}{break}{/if}
{list a as x}
{var first_option =}
{list x.voteDetailList as voteToOption}
{if voteToOption==1}
{if first_option==false},{/if}&&“${b[voteToOption_index]}”&&
{if (x.role!="-1") },“我是${c[x.role]}”&&{/if}
&&&&&&&&${fn1(x.voteTime)}
{if x.userName==''}{/if}
网易公司版权所有&&
{list x.l as y}
{if defined('wl')}
{list wl as x}{/list}&&&&hadoop-0.20.2-core eclipse jar包
&hadoop-0.20.2-core eclipse jar包
hadoop-0.20.2-core eclipse jar包
若举报审核通过，可奖励20下载分
被举报人：
cainiaoxuexi
举报的资源分：
请选择类型
资源无法下载
资源无法使用
标题与实际内容不符
含有危害国家安全内容
含有反动色情等内容
含广告内容
版权问题，侵犯个人或公司的版权
*详细原因：
VIP下载&&免积分60元/年（1200次）
您可能还需要
Q.为什么我点的下载下不了，但积分却被扣了
A. 由于下载人数众多，下载服务器做了并发的限制。若发现下载不了，请稍后再试，多次下载是不会重复扣分的。
Q.我的积分不多了，如何获取积分？
A. 获得积分，详细见。
完成任务获取积分。
论坛可用分兑换下载积分。
第一次绑定手机，将获得5个C币，C币可。
关注并绑定CSDNID，送10个下载分
下载资源意味着您已经同意遵守以下协议
资源的所有权益归上传用户所有
未经权益所有人同意，不得将资源中的内容挪作商业或盈利用途
CSDN下载频道仅提供交流平台，并不能对任何下载资源负责
下载资源中如有侵权或不适当内容，
本站不保证本站提供的资源的准确性，安全性和完整性，同时也不承担用户因使用这些下载资源对自己和他人造成任何形式的伤害或损失。
开发技术下载排行
您当前C币：0&&&可兑换 0 下载积分
兑换下载分:&
消耗C币:0&
立即兑换&&
兑换成功你当前的下载分为 。前去下载资源
你下载资源过于频繁，请输入验证码
如何快速获得积分？
你已经下载过该资源，再次下载不需要扣除积分
hadoop-0.20.2-core eclipse jar包
所需积分：1
剩余积分：0
扫描微信二维码精彩活动、课程更新抢先知
VIP会员，免积分下载
会员到期时间：日
剩余下载次数：1000
hadoop-0.20.2-core eclipse jar包
剩余次数：&&&&有效期截止到：
你还不是VIP会员VIP会员享免积分 . 专属通道极速下载
VIP下载次数已满VIP会员享免积分 . 专属通道极速下载，请继续开通VIP会员
你的VIP会员已过期VIP会员享免积分 . 专属通道极速下载，请继续开通VIP会员