• Staff/Senior Data Engineer - Big Data & Cloud Jobs in Bangalore,India

  • Nastech Consulting

Job Description:


A data engineer is a key contributor in preparing and making data available in a safe and secure manner, for multiple usages by Businesses and Functions, and for multiple purposes reporting, analytics and/or applications The individual contributes to potentially big productivity for all Data Consumers ie people consuming or using the data - of the Businesses and Functions, by delivering well designed and running Data Assets

Key Responsibilities

- To design, develop, enhance enterprise applications using Big Data technologies to meet business process and application requirements

- To design and develop the logical and physical- data models and data stores, and to industrialise the feeding and deliveries from the integrated datastores (data warehouse, DataMart, data sets)

- To streamline these data pipelines across the enterprise, for maximum usage throughout the company or business/function

- Prior client facing skills

- Experience working on Banking and Capital Markets

- Willingness to travel to meet client needs

- To design and develop the API providing access to the data

- To prototype and build visualisations on top of the data, allowing to create insights, report and/or take fact-based decisions Potentially collaborate with Data Visualisation experts in delivering the right data/data set

- Build infrastructure optimal extraction, transformation and loading of data from a wide variety of data sources using Spark and programming languages such as Python or Scala

- Ability to follow Agile methodology

- Provide product and design level technical best practices

- Work with theteam to build innovative, robust, and easy-to-use features

- Building a quality culture

- Foster teamwork and lead by example

- Participating in internal practice development initiatives

- Need to work as a team member to contribute in various technical streams of Data and Analytics implementation project

- Quality review of detailed design and code provided by external suppliers

- Documentation of activities at all stages of SDLC

Qualifications BE/BTech/MCA/MBA with 6+yearsindustry experience

Experience (eg type of work, projects, area of business etc)

Technical skillsets

- Experience working on Hadoop distribution systems (Cloudera /Hortonworks/MapR)Role

- Strong understanding & familiarity with all Hadoop Ecosystem components and Hadoop Administrative Fundamentals

- Strong understanding of underlying Hadoop Architectural concepts and distributed computing paradigms

- Experience in the development of Hadoop APIs and MapReduce jobs for large scale data processing

- Hands-on programming experience in Apache Spark using SparkSQL and Spark Streaming or Apache Storm

- Data Integration experience on Kafka, Sqoop, Nifi, Flink, Talend, Informatica, DataStage etc

- Hands on experience with major components like Hive, PIG, Spark, MapReduce

- Hands on experience with at-least one of the Cloud service platform from Microsoft Azure (Console and SDK) or Amazon AWS

- Experience in NOSQL technologies like Hbase, Cassandra, MongoDB

- Experience in ETL on Azure /AWS cloud and Databricks

- Knowledge of datamodelling tools (Power designer, Erwin, or other accepted standard tool),CI/CD Tools (Jenkins, Marvin, GIT)

- Knowledge in big data architecture patterns [Lambda, Kappa]

- Strong proficiency in using query languages such as SQL, Hive and SparkSQL

- Experience in Python, Spark and Scala programming languages

- Should have worked on Databases (MySQL, Oracle, MSSQL, SQL Server 2008/2012)

- Good knowledge of ShellScripting and Linux

- Knowledge of SDLC, Scrum, Agile methodologies

Behavioral/Domain skillsets

- Prior client facing skills

- Experience working on Banking and Capital Markets

- Willingness to travel to meet client needs

- Experience in converting business problems/challenges to technical solutions considering security, performance, scalability etc

- Experience in enterprise grade solution implementations

- Multi-tasking attitude, equally comfortable operating from within both the technology & business environments

- Flexible and proactive/self-motivated working style with strong personal ownership of problem resolution excellent inter-personal skills

- Ability to multi-task under pressure and work independently with minimal supervision

- Self-starter who can work effectively with minimal day-to-day supervision

- Strong problem-solving skills & proactive attitude

- Client focused with good presentation, communication and relationship building skills

- Up-to-date knowledge of the latest industry regulations and have a keen interest in future technology trends

- Should be able to identify profitable opportunities, sell-on at client work, make a significant contribution in supporting a market proposition/capability and wider practice contribution

Profile Summary:

Employment Type : Full Time
Salary : Not Mentioned
Deadline : 21st Mar 2020

Key Skills:

Company Profile:

Not Mentioned

Would you like to try out these free online tutorials?

People who search this job also searched for the following Keywords

Salary trends based on over 1 crore profiles

View Salaries

All rights reserved © 2018 Wisdom IT Services India Pvt. Ltd DMCA.com Protection Status