Phillip Wēi's Blog

Click on 'ABOUT ME' on the top right, you will find more about this idiot :)

Accounting Projects

ANU_BUSN_COURSES_2019

ANU Accounting Courses Group Assignment Projects This semester is quite busy for me because apart from backend development learning, I also have three accounting research projects at the same...

Posted by Phillip on October 15, 2019

MapReduce的shuffle过程

shuffle过程详解

前言 MapReduce作为Hadoop的编程框架，对于大数据开发或者想要接触大数据开发的学习者来说，是必须要掌握的。MapReduce计算模型主要由三个阶段构成：Map、shuffle、Reduce。Map是映射，负责数据的过滤分法，将原始数据转化为键值对；Reduce是合并，将具有相同key值的value进行处理后再输出新的键值对作为最终结果。为了让Reduce可以并行处理Map的结果...

Posted by Phillip on March 12, 2016

Hadoop 完全分布式搭建

集群规划 → 不同的守护进程运行哪些节点

结构 HDFS: Namenode SecndaryNamenode Datanode Datanode Datanode Yarn: Nodemanager Nodemanager Nodemanager ResourceManager Histrory: HistroryServer 基本环境准备对于一个初学者来说，如果想要对海量数据进行...

Posted by Phillip on January 19, 2016

FEATURED TAGS

Hadoop Big Data

ABOUT ME

An enthusiastic full stack learner who is studying accounting.

✉️ Phillip.Xing@cn.ey.com

FRIENDS