Kinesis or Kafka System for Realtime Processing

We are going to be processing Realtime market data (stocks, etc.) and need to write to an in-memory database (or could use Kafka for this, or?) for quick reads to web and mobile clients as well as piping the data to either amazon S3 or other disk storage. A set of alerts and rules will need to be run on the incoming data (e.g. price X > Y) and we suspect that we will be using a myriad of AWS products to send notifications, write to logs, etc. Experience with websockets together with kinesis (i.e. write from kinesis to a websocket stack) would be a plus.

Were on the fence between Kafka and Kinesis. Heroku offers a managed Kafka service that would be nice, but overall I am leaning towards kinesis because of the full management services, and the integration in SDKs and with other AWS services. Could be convinced otherwise, though.

Looking for someone with experience the can guide and educate as well as help implement. Possibility this contract could be in perpetuity if system proves of use to the stakeholders.
About the recuiter
Member since Nov 11, 2022
Xx Xx
from Mexico City, Mexico

---

---

Looking for help? Checkout our video tutorial
How to search and apply for jobs

How to apply? Do you have more questions about the Job?
See frequently asked questions