Python with spark project

Posted by - Hr T. at Nov 25, 2021
Tamil Nadu, India

BudgetTypeStatusValid Through
$19.83HOURLYOpenDec 25, 2021

Job Description

Senior Data Engineer

Contract duration - 3 month

TheDataTeam is looking for a Senior Data Engineer to join and contribute to the

engineering consulting services of the company.

The Senior Data Engineer role is primarily responsible for building out entire engineering artifacts (code and documentation) for clients delivered on top of the Hadoop ecosystem.

The Responsibilities Of the Senior Data Engineer Are

brand and values

A Candidate Must Have The Following Skills:

Should have good code review and debugging skills

Bachelor’s in Computer Science, and preferably, a Master’s as well

(Mysql/Oracle/MongoDB/Postgres) preferable Postgres

Should be well versed with at least one Database

Strong understanding of various storage formats (ORC/Parquet/Avro)

Should have experience in full life-cycle of one big data application

Experience in Cloud environment (GCP/AWS/Azure) preferably AWS

Minimum 4 years of experience

as management tasks, and see them to completion with minimal guidance

Detail oriented with willingness to proactively own software tasks as well

Exposure and commitment to agile methodologies

Excellent software design and implementation ability

Strong problem-solving ability

Manage stakeholders with proactive communication upholding TheDataTeam

Mentoring juniors in a dynamic team setting

MapReduce, Spark, Hive, using Python for application design.

Implement the pipelines using execution frameworks including but not limited to

boards to build towards, using a test-driven approach

Breakdown complexity and translate requirements to specification artifacts and story

Requirements gathering and assessment, understand existing Ruby scripts


Skills & Expertise Required

Python Spark Pyspark Azure AWS 

Offer to work on this project closes in 19 days!

Similar Projects

.Net developer with Azure experienc...

Looking for an experienced .NET developer that can assist in architecture, implementation, and Azure administration for a RESTful application with a mobile app written in Cordova

Hourly, $13.89

Azure web portal and chat framework

Looking for an experience Azure WebApp developer to creat a SaaS platform with client web portal, chat and AI framework, desktop and mobile app functions. API connectivity to 3rd party platforms such...read more
Hourly, $13.89

Airflow Consulting

Looking for an airflow expert to guide my ETL project workflow (get data from rest and database).
Hourly, $21.60

Hr hiring Python, Spark, Pyspark, Azure, AWS experts from & nearby:

  • Afghanistan
  • Albania
  • Algeria
  • American Samoa
  • Andorra
  • Angola
  • Anguilla
  • Antarctica
  • Antiguaand Barbuda
  • Argentina
  • Armenia
  • Aruba
  • Australia
  • Austria
  • Azerbaijan
  • Bahamas
  • Bahrain
  • Bangladesh
  • Barbados
  • Belarus
  • Belgium
  • Belize
  • Benin
  • Bermuda
  • Bhutan
  • Bolivia
  • Bosniaand Herzegovina
  • Botswana
  • Bouvet Island
  • Brazil
  • British Indian Ocean Territory
  • Brunei
  • Bulgaria
  • Burkina Faso
  • Burundi
  • Cambodia
  • Cameroon
  • Canada
  • Cape Verde
  • Cayman Islands
  • Central African Republic
  • Chad
  • Chile
  • China
  • Christmas Island
  • Cocos Islands
  • Colombia
  • Comoros
  • Republic of the Congo
  • Democratic Republicof the Congo
  • Cook Islands
  • Costa Rica
  • Ivory Coast
  • Croatia
  • Cuba
  • Cyprus
  • Czech Republic
  • Denmark
  • Djibouti
  • Dominica
  • Dominican Republic
  • EastTimor
  • Ecuador
  • Egypt
  • El Salvador
  • Equatorial Guinea
  • Eritrea
  • Estonia
  • Ethiopia
  • External Territories of Australia
  • Falkland Islands
  • Faroe Islands
  • Fiji
  • Finland
  • France
  • French Guiana
  • French Polynesia
  • French Southern Territories
  • Gabon
  • Gambia
  • Georgia
  • Germany
  • Ghana
  • Gibraltar
  • Greece
  • Greenland
  • Grenada
  • Guadeloupe
  • Guam
  • Guatemala
  • Guernseyand Alderney
  • Guinea
  • Guinea-Bissau
  • Guyana
  • Haiti
  • Heard Islandand McDonald Islands
  • Honduras
  • HongKong
  • Hungary
  • Iceland
  • India
  • Indonesia
  • Iran
  • Iraq
  • Ireland
  • Israel
  • Italy
  • Jamaica
  • Japan
  • Jersey
  • Jordan
  • Kazakhstan
  • Kenya
  • Kiribati
  • North Korea
  • South Korea
  • Kuwait
  • Kyrgyzstan
  • Laos
  • Latvia
  • Lebanon
  • Lesotho
  • Liberia
  • Libya
  • Liechtenstein
  • Lithuania
  • Luxembourg
  • Macao
  • Macedonia
  • Madagascar
  • Malawi
  • Malaysia
  • Maldives
  • Mali
  • Malta
  • Man(Isleof)
  • Marshall Islands
  • Martinique
  • Mauritania
  • Mauritius
  • Mayotte
  • Mexico
  • Micronesia
  • Moldova
  • Monaco
  • Mongolia
  • Montserrat
  • Morocco
  • Mozambique
  • Myanmar
  • Namibia
  • Nauru
  • Nepal
  • Netherlands Antilles
  • Netherlands
  • New Caledonia
  • New Zealand
  • Nicaragua
  • Niger
  • Nigeria
  • Niue
  • Norfolk Island
  • Northern Mariana Islands
  • Norway
  • Oman
  • Pakistan
  • Palau
  • Palestinian Territory
  • Panama
  • Papua New Guinea
  • Paraguay
  • Peru
  • Philippines
  • Pitcairn
  • Poland
  • Portugal
  • Puerto Rico
  • Qatar
  • Reunion
  • Romania
  • Russia
  • Rwanda
  • Saint Helena
  • Saint Kittsand Nevis
  • Saint Lucia
  • Saint Pierreand Miquelon
  • Saint Vincentandthe Grenadines
  • Samoa
  • San Marino
  • Sao Tomeand Principe
  • Saudi Arabia
  • Senegal
  • Serbia
  • Seychelles
  • SierraLeone
  • Singapore
  • Slovakia
  • Slovenia
  • Smaller Territories of the UK
  • Solomon Islands
  • Somalia
  • South Africa
  • South Georgia and the South Sandwich Islands
  • South Sudan
  • Spain
  • Sri Lanka
  • Sudan
  • Suriname
  • Svalbardand Jan Mayen
  • Swaziland
  • Sweden
  • Switzerland
  • Syria
  • Taiwan
  • Tajikistan
  • Tanzania
  • Thailand
  • Togo
  • Tokelau
  • Tonga
  • Trinidadand Tobago
  • Tunisia
  • Turkey
  • Turkmenistan
  • Turksand Caicos Islands
  • Tuvalu
  • Uganda
  • Ukraine
  • United Arab Emirates
  • United Kingdom
  • United States
  • United States Minor Outlying Islands
  • Uruguay
  • Uzbekistan
  • Vanuatu
  • Vatican
  • Venezuela
  • Vietnam
  • British Virgin Islands
  • U.S.Virgin Islands
  • Wallis and Futuna
  • Western Sahara
  • Yemen
  • Yugoslavia
  • Zambia
  • Zimbabwe
  • Aland Islands
  • Saint Barthélemy
  • Bonaire, Saint Eustatiusand Saba
  • Curaçao
  • Guernsey
  • Isleof Man
  • Jersey
  • Montenegro
  • Saint Martin
  • Sint Maarten
  • East Timor
  • Kosovo