CSV files are generated in other processes. Python script is needed to upload the data from these files into a Bigquery table.
Script Language:
Python
Scrip should do:
The script is run in a specific folder.
Scan that folder for config file -- load that.
Scan that folder for CSV files
If a file found then map columns ( fields ) between CSV fields and the bigquery table.
Then push the CSV data into bigquery table.
Once the data was uploaded, the CSV file should be moved into a sub directory.
Repeat the process until all the files were pushed into the Bigquery table.
Config file should include
Bigquery table details
Mapping from CSV fields to the table fields
Resources provided:
Bigquery: access to a sample table
CSV files: A number of files will be provided
The final product should allow us to:
Select the destination table in Bigquery
Select csv source folder in Drive
About the recuiterMember since Mar 14, 2020 Pine Host
from Oost-Vlaanderen, Belgium