Find more Data Mining And Management Remote Jobs posted recently Worldwide

Required Amazon S3,Amazon Web Services,Apache Spark,Pyspark,Python freelancer for Converting JSON or Avro files to Parquet job

Posted at - Sep 28, 2022

Toogit Instant Connect Enabled

I need to convert JSON, Avro or other row-based format files in S3 into Parquet columnar store formats using an AWS service like EMR or Glue.

I already have code that converts JSON to parquet using Python but the process is very manual, accounting for NULL values in the JSON elements by looking at each and every field/column and putting in default values if there's a NULL.

I am looking for an easier, less manual way of doing this using something like Spark or other similar methods.

Since I am working exclusively on AWS, I am only looking for solutions using AWS services such as EMR, Glue or similar AWS service.

I am thus looking for someone with experience using AWS EMR, Glue, Python, Pyspark etc.

Please note: Since this is going to be a learning experience for me, this is going to be a live session on Skype, Zoom, Google Hangouts etc where you code and I watch and you answer any questions I have in the process.

Thus, I will pay in one-hour increments. The initial contract is going to be for one hour and if we need more time we can have another one hour contract and so on and so forth.

Please only apply if you're ok with all these conditions and have the required experience.

About the recuiterMember since Jul 6, 2017 Kieran
from Bergamo, Italy

Skills & Expertise Required

Amazon S3 Amazon Web Services Apache Spark Pyspark Python 

Candidate shortlisted and hiredHiring open till - Oct 19, 2022

Work from Anywhere
40 hrs / week
Hourly Type
Remote Job

Looking for help? Checkout our video tutorial
How to search and apply for jobs

How to apply? Do you have more questions about the Job?
See frequently asked questions

Apply on more work from home jobs posted in Data Mining And Management category.