Suraj Nayak's Blog

What Young Indians must be doing right now

It's so much our prime minister Modi is trying to do. He always stress the fact that - India has 65% of population aging less than 35. These numbers screams at me something. What other countries don't have and we Indians have. Irrespective of

data-science

Data Science Learning Path & Experience

The path from Data Engineer to Data Science Engineer may not be very easy. But, with constant effort and dedication one can be a Data Science Engineer. When I started the path towards this new area there were hundreds of questions that made me

Inspiration

My Suggestions Which May Be Helpful For Others

Success is a journey, not a destination. The doing is often more important than the outcome. -Arthur Ashe Source When I meet my friends, (former) colleagues and relatives who are studying or working, frequently ask suggestions or question on what can be done to

hadoop

A VM for newbie Hadoop contributors

Following Arnold's 6 Rules, this is a blog for Give something back. Opensource community has contributed a lot of frameworks with which many of our projects can reuse the work contributed by the community. Thus helping organizations to get started right away from the

Hadoop Code - Deep Dive Series - Part 3 - File System XAttrs

Hadoop Common hosts enums for Extended Attributers. Extended attributes is an amazing feature of HDFS file system. So I thought of making a deep dive into XAttr. Let's start with what official documentation says: Extended attributes (abbreviated as xattrs) are a filesystem feature that

Hadoop Code - Deep Dive Series - Part 2 - High Availabilty

Before I continue the deep dive series, I want to thank Apache Hadoop community in making sure following key aspects are taken care: The code is commented well Test cases are written to cover the features Checkstyle coding standards enforcers Code becomes document and

Hadoop Code - Deep Dive Series - Part 1 - Introduction

To better understand hadoop, it is always good to know the internals. With this motto, I started exploring hadoop code base (2.7.3 branch) starting with hadoop-common module in the hadoop-main parent project. Disclaimer: This blog post is not for Hadoop beginners. This

Setup Apache Kafka on Windows System

Hi Learners, Here, I am going to show how to set apache kafka on windows machine. Before starting let understand the basic terminology about Apache kafka. Kafka- Kafka is a distributed messaging system providing fast, highly scalable and redundant messaging through a publish-subscribe pattern

Simple way to join two csv files in Spark

Today, I will show you a very simple way to join two csv files in Spark. In one of our Big Data / Hadoop projects, we needed to find an easy way to join two csv file in spark. We explored a lot of techniques

Introd Oozie : How to Run Shell script in Oozie using SSH action

How to Run Shell script in Oozie using SSH action Apache Hadoop is open source framework. There are so many jobs running in different time framework in hadoop ecosystem. there are some open source job scheduler hadoop job scheduler. Oozie Azkaban Oozie is also

Elasticsearch

Explore Elasticsearch and Lucene-Solr : Day 1

Search Enabling search unleash great insights on the data you have. You also empower users to get to the right data what they are looking for. I thought of exploring open source frameworks. Here are the two frameworks I plan to explore. [Lucene-Solr](http: