Find more Data Mining And Management Remote Jobs posted recently Worldwide

Required Amazon S3,Amazon Web Services,Apache Spark,Pyspark,Python freelancer for Converting JSON or Avro files to Parquet job

Posted at - May 3, 2019

I need to convert JSON, Avro or other row-based format files in S3 into Parquet columnar store formats using an AWS service like EMR or Glue.

I already have code that converts JSON to parquet using Python but the process is very manual, accounting for NULL values in the JSON elements by looking at each and every field/column and putting in default values if there's a NULL.

I am looking for an easier, less manual way of doing this using something like Spark or other similar methods.

Since I am working exclusively on AWS, I am only looking for solutions using AWS services such as EMR, Glue or similar AWS service.

I am thus looking for someone with experience using AWS EMR, Glue, Python, Pyspark etc.

Please note: Since this is going to be a learning experience for me, this is going to be a live session on Skype, Zoom, Google Hangouts etc where you code and I watch and you answer any questions I have in the process.

Thus, I will pay in one-hour increments. The initial contract is going to be for one hour and if we need more time we can have another one hour contract and so on and so forth.

Please only apply if you're ok with all these conditions and have the required experience.

About the recuiterMember since May 20, 2018 Dibyashakti Moh
from Karnataka, India

Skills & Expertise Required

Amazon S3 Amazon Web Services Apache Spark Pyspark Python 

Candidate shortlisted and hiredHiring open till - Feb 27, 2020

Work from Anywhere
40 hrs / week
Hourly Type
Remote Job

Looking for help? Checkout our video tutorial
How to search and apply for jobs

How to apply? Do you have more questions about the Job?
See frequently asked questions

Apply on more work from home jobs posted in Data Mining And Management category.

Related Jobs

Latest In Amazon S3 Jobs

Latest In Amazon Web Services Jobs

Latest In Apache Spark Jobs

Latest In Pyspark Jobs

Latest In Python Jobs