Find more Data Mining And Management Remote Jobs posted recently Worldwide

Required Amazon S3,Amazon Web Services,Apache Spark,Pyspark,Python freelancer for Converting JSON or Avro files to Parquet job

Posted at - Feb 10, 2024

Toogit Instant Connect Enabled


I need to convert JSON, Avro or other row-based format files in S3 into Parquet columnar store formats using an AWS service like EMR or Glue.

I already have code that converts JSON to parquet using Python but the process is very manual, accounting for NULL values in the JSON elements by looking at each and every field/column and putting in default values if there's a NULL.

I am looking for an easier, less manual way of doing this using something like Spark or other similar methods.

Since I am working exclusively on AWS, I am only looking for solutions using AWS services such as EMR, Glue or similar AWS service.

I am thus looking for someone with experience using AWS EMR, Glue, Python, Pyspark etc.

Please note: Since this is going to be a learning experience for me, this is going to be a live session on Skype, Zoom, Google Hangouts etc where you code and I watch and you answer any questions I have in the process.

Thus, I will pay in one-hour increments. The initial contract is going to be for one hour and if we need more time we can have another one hour contract and so on and so forth.

Please only apply if you're ok with all these conditions and have the required experience.

About the recuiterMember since Nov 11, 2022 Pankaj Doot
from Gandaria, Indonesia

Skills & Expertise Required

Amazon S3 Amazon Web Services Apache Spark Pyspark Python 

Open for hiringApply before - Aug 8, 2024

Work from Anywhere
40 hrs / week
Hourly Type
Remote Job
$34.49
Cost

Offer to work on this project closes in 110 days!
Are you interested in this Opportunity?

Apply Now

Looking for help? Checkout our video tutorial
How to search and apply for jobs

How to apply? Do you have more questions about the Job?
See frequently asked questions


Apply on more work from home jobs posted in Data Mining And Management category.


Related Jobs


Latest In Amazon S3 Jobs


Latest In Amazon Web Services Jobs


Latest In Apache Spark Jobs


Latest In Pyspark Jobs


Latest In Python Jobs