We are going to be processing realtime market data (stocks, etc) and need to write to an in-memory database (or could use kafka for this, or?) for quick reads to web and mobile clients as well as piping the data to either amazon S3 or other disk storage. A set of alerts and rules will need to be run on the incoming data (e.g. price X > Y) and we suspect that we will be using a myriad of AWS products to send notifications, write to logs, etc. Experiecne with websockets together with kinesis (i.e. write from kinesis to a websocket stack) would be a plus.
We're on the fence between Kafka and Kinesis. Heroku offers a managed kafka service that would be nice, but overall I am leaning towards kinesis because of the full management services, and the integration in SDKs and with other AWS services. Could be convinced otherwise, though.
Looking for someone with experience the can guide and educate as well as help implement. Possibility this contract could be in perpetuity if system proves of use to the stakeholders.
About the recuiterMember since Sep 2, 2017 Owen P.
from Texas, United States