Description: Hadoop是一个用于运行应用程序在大型集群的廉价硬件设备上的框架。Hadoop为应用程序透明的提供了一组稳定/可靠的接口和数据运动。在 Hadoop中实现了Google的MapReduce算法，它能够把应用程序分割成许多很小的工作单元，每个单元可以在任何集群节点上执行或重复执行。此外，Hadoop还提供一个分布式文件系统用来在各个计算节点上存储数据，并提供了对数据读写的高吞吐率。由于应用了map/reduce和分布式文件系统使得Hadoop框架具有高容错性，它会自动处理失败节点。已经在具有600个节点的集群测试过Hadoop框架。- Apache Hadoop Core is a software platform that lets one easily write and run applications that process vast amounts of data.
Here s what makes Hadoop especially useful:
* Scalable: Hadoop can reliably store and process petabytes.
* Economical: It distributes the data and processing across clusters of commonly available computers. These clusters can number into the thousands of nodes.
* Efficient: By distributing the data, Hadoop can process it in parallel on the nodes where the data is located. This makes it extremely rapid.
* Reliable: Hadoop automatically maintains multiple copies of data and automatically redeploys computing tasks based on failures.
Hadoop implements MapReduce, using the Hadoop Distributed File System (HDFS) (see figure below.) MapReduce divides applications into many small blocks of work. HDFS creates multiple replicas of data blocks for reliability, placing them on compute nodes around the cluster. MapReduce can then process the data w Platform: |
Size: 3598336 |
Author:宾利金 |
Hits:
Description: In this text, students of applied mathematics, science and engineering are introduced to fundamental ways of thinking about the broad context of parallelism. The authors begin by giving the reader a deeper understanding of the issues through a general examination of timing, data dependencies, and communication. These ideas are implemented with respect to shared memory, parallel and vector processing, and distributed memory cluster computing. Threads, OpenMP, and MPI are covered, along with code examples in Fortran, C, and Java. The principles of parallel computation are applied throughout as the authors cover traditional topics in a first course in scientific computing. Building on the fundamentals of floating point representation and numerical error, a thorough treatment of numerical linear algebra and eigenvector/eigenvalue problems is provided. By studying how these algorithms parallelize, the reader is able to explore parallelism inherent in other computations, such as Monte Carlo methods Platform: |
Size: 1644544 |
Author:zahid |
Hits:
Description: Parallel Java是基于Java提供的并行编程API，其目标是（1）在一套API中同时支持基于线程/共享内存的并行编程和基于消息/集群的并行编程；（2）提供与OpenMP和MPI相同的功能，但采用面向对象的Java API；和（3）很容易部署在异构计算环境中，包括单核CPU、多核CPU及集群。本文介绍了Parallel Java的特点和体系结构，并与其他基于Java的并行开发中间件进行了对比。(Parallel Java is a API parallel programming based on Java provides the goal (1) in a API and thread support / shared memory parallel programming and parallel programming based on message / cluster; (2) with OpenMP and MPI provide the same functionality, but the Java API object oriented; and (3) is easy to deploy in a heterogeneous computing environment, including single core CPU, and multi-core CPU cluster. This paper introduces the characteristics and architecture of Parallel Java, and compares it with other Java based parallel development middleware.) Platform: |
Size: 107520 |
Author:kmsj
|
Hits:
Description: Hadoop是一个开发和运行处理大规模数据的软件平台,是Appach的一个用java语言实现开源软件框架，实现在大量计算机组成的集群中对海量数据进行分布式计算.(Hadoop is a development and operation of large-scale data processing software platform is a Appach using java language to achieve the realization of open source software framework, consisting of a large number of computer cluster distributed computing for massive data.) Platform: |
Size: 2151424 |
Author:求知鸟
|
Hits:
Description: spark大数据，这是一本介绍spark相关知识的好书籍，希望大家喜欢！(Apache Spark is a fast and general-purpose cluster computing system. It provides high-level APIs in Java, Scala, Python and R, and an optimized engine that supports general execution graphs. It also supports a rich set of higher-level tools including Spark SQL for SQL and structured data processing, MLlib for machine learning, GraphX for graph processing, and Spark Streaming.) Platform: |
Size: 5912576 |
Author:jiangchuang123 |
Hits:
Search - java cluster computing