Kinesis or Kafka System for Realtime Processing

Posted by - Owen P. at Apr 9, 2021
Texas, United States

BudgetTypeStatusValid Through
$775.19FIXEDOpenApr 30, 2021

Job Description

We are going to be processing realtime market data (stocks, etc) and need to write to an in-memory database (or could use kafka for this, or?) for quick reads to web and mobile clients as well as piping the data to either amazon S3 or other disk storage. A set of alerts and rules will need to be run on the incoming data (e.g. price X > Y) and we suspect that we will be using a myriad of AWS products to send notifications, write to logs, etc. Experiecne with websockets together with kinesis (i.e. write from kinesis to a websocket stack) would be a plus.

We're on the fence between Kafka and Kinesis. Heroku offers a managed kafka service that would be nice, but overall I am leaning towards kinesis because of the full management services, and the integration in SDKs and with other AWS services. Could be convinced otherwise, though.

Looking for someone with experience the can guide and educate as well as help implement. Possibility this contract could be in perpetuity if system proves of use to the stakeholders.

Skills & Expertise Required

Apache Kafka AWS IoT Core 

Offer to work on this project closes in 8 days!

Try more jobs by Owen P.

Looking for Hardware Help - Getting a 5G Connection - Fixed
Shopify store creation - Hourly
Azure Machine Learning expert - Hourly
Need Electrical Engineer for Audio Compressor - Hourly
5 Page WordPress Site Needed - Hourly
We need trainers who can deliver our courses online - Hourly
I would like to develop an ecommerce trading application, with functionality similar to a dating app - Fixed
"Jack of all Admin trades" extremely organized. Real Estate Exp a PLUS! - Hourly
Python-based software architect needed to consult on web app development - Hourly
Firmware and Interface for PCB - Fixed

Similar Projects

Apache Spark Streaming with Kafka

Looking for an experienced coder, who can write code efficiently in Java and Scala for Apache Spark Streaming. He should also have a strong knowledge of kafka and should be able to create a data pipel...read more
Hourly, $21.71

Terratest Smoke Test Integration to...

To integrate Terratest smoke test into the CI/CD pipeline, with knowledge of Kubernetes, Terraform, Terratesting, Go Language, Kafka

Smoke test Cases:
Kubernetes is up and running
All...read more
Hourly, $15.50

Remote Senior JAVA Developer needed

Lviv IT company URGENTLY looking for remote work Senior JAVADeveloper for the project.
At least 3 years of commercial experience;
Tech stack Java, Amazon SES; Spring (Core, MVC, Security, Dat...read more
Hourly, $13.95

Owen hiring Apache Kafka, AWS IoT Core experts from & nearby:

  • Afghanistan
  • Albania
  • Algeria
  • American Samoa
  • Andorra
  • Angola
  • Anguilla
  • Antarctica
  • Antiguaand Barbuda
  • Argentina
  • Armenia
  • Aruba
  • Australia
  • Austria
  • Azerbaijan
  • Bahamas
  • Bahrain
  • Bangladesh
  • Barbados
  • Belarus
  • Belgium
  • Belize
  • Benin
  • Bermuda
  • Bhutan
  • Bolivia
  • Bosniaand Herzegovina
  • Botswana
  • Bouvet Island
  • Brazil
  • British Indian Ocean Territory
  • Brunei
  • Bulgaria
  • Burkina Faso
  • Burundi
  • Cambodia
  • Cameroon
  • Canada
  • Cape Verde
  • Cayman Islands
  • Central African Republic
  • Chad
  • Chile
  • China
  • Christmas Island
  • Cocos Islands
  • Colombia
  • Comoros
  • Republic of the Congo
  • Democratic Republicof the Congo
  • Cook Islands
  • Costa Rica
  • Ivory Coast
  • Croatia
  • Cuba
  • Cyprus
  • Czech Republic
  • Denmark
  • Djibouti
  • Dominica
  • Dominican Republic
  • EastTimor
  • Ecuador
  • Egypt
  • El Salvador
  • Equatorial Guinea
  • Eritrea
  • Estonia
  • Ethiopia
  • External Territories of Australia
  • Falkland Islands
  • Faroe Islands
  • Fiji
  • Finland
  • France
  • French Guiana
  • French Polynesia
  • French Southern Territories
  • Gabon
  • Gambia
  • Georgia
  • Germany
  • Ghana
  • Gibraltar
  • Greece
  • Greenland
  • Grenada
  • Guadeloupe
  • Guam
  • Guatemala
  • Guernseyand Alderney
  • Guinea
  • Guinea-Bissau
  • Guyana
  • Haiti
  • Heard Islandand McDonald Islands
  • Honduras
  • HongKong
  • Hungary
  • Iceland
  • India
  • Indonesia
  • Iran
  • Iraq
  • Ireland
  • Israel
  • Italy
  • Jamaica
  • Japan
  • Jersey
  • Jordan
  • Kazakhstan
  • Kenya
  • Kiribati
  • North Korea
  • South Korea
  • Kuwait
  • Kyrgyzstan
  • Laos
  • Latvia
  • Lebanon
  • Lesotho
  • Liberia
  • Libya
  • Liechtenstein
  • Lithuania
  • Luxembourg
  • Macao
  • Macedonia
  • Madagascar
  • Malawi
  • Malaysia
  • Maldives
  • Mali
  • Malta
  • Man(Isleof)
  • Marshall Islands
  • Martinique
  • Mauritania
  • Mauritius
  • Mayotte
  • Mexico
  • Micronesia
  • Moldova
  • Monaco
  • Mongolia
  • Montserrat
  • Morocco
  • Mozambique
  • Myanmar
  • Namibia
  • Nauru
  • Nepal
  • Netherlands Antilles
  • Netherlands
  • New Caledonia
  • New Zealand
  • Nicaragua
  • Niger
  • Nigeria
  • Niue
  • Norfolk Island
  • Northern Mariana Islands
  • Norway
  • Oman
  • Pakistan
  • Palau
  • Palestinian Territory
  • Panama
  • Papua New Guinea
  • Paraguay
  • Peru
  • Philippines
  • Pitcairn
  • Poland
  • Portugal
  • Puerto Rico
  • Qatar
  • Reunion
  • Romania
  • Russia
  • Rwanda
  • Saint Helena
  • Saint Kittsand Nevis
  • Saint Lucia
  • Saint Pierreand Miquelon
  • Saint Vincentandthe Grenadines
  • Samoa
  • San Marino
  • Sao Tomeand Principe
  • Saudi Arabia
  • Senegal
  • Serbia
  • Seychelles
  • SierraLeone
  • Singapore
  • Slovakia
  • Slovenia
  • Smaller Territories of the UK
  • Solomon Islands
  • Somalia
  • South Africa
  • South Georgia and the South Sandwich Islands
  • South Sudan
  • Spain
  • Sri Lanka
  • Sudan
  • Suriname
  • Svalbardand Jan Mayen
  • Swaziland
  • Sweden
  • Switzerland
  • Syria
  • Taiwan
  • Tajikistan
  • Tanzania
  • Thailand
  • Togo
  • Tokelau
  • Tonga
  • Trinidadand Tobago
  • Tunisia
  • Turkey
  • Turkmenistan
  • Turksand Caicos Islands
  • Tuvalu
  • Uganda
  • Ukraine
  • United Arab Emirates
  • United Kingdom
  • United States
  • United States Minor Outlying Islands
  • Uruguay
  • Uzbekistan
  • Vanuatu
  • Vatican
  • Venezuela
  • Vietnam
  • British Virgin Islands
  • U.S.Virgin Islands
  • Wallis and Futuna
  • Western Sahara
  • Yemen
  • Yugoslavia
  • Zambia
  • Zimbabwe
  • Aland Islands
  • Saint Barthélemy
  • Bonaire, Saint Eustatiusand Saba
  • Curaçao
  • Guernsey
  • Isleof Man
  • Jersey
  • Montenegro
  • Saint Martin
  • Sint Maarten
  • East Timor
  • Kosovo