java - Can i use RabbitMQ to move data to Amazon Kinesis stream? -

java - Can i use RabbitMQ to move data to Amazon Kinesis stream? -

i have server containing folders date wise , each folder farther contains many files (size 200kb each) containing log particular day. new rabbitmq , while going through documentation of rabbitmq found below code producer

refer link: https://github.com/rabbitmq/rabbitmq-tutorials/blob/master/java/send.java

public class send { private final static string queue_name = "hello"; public static void main(string[] argv) throws exception { connectionfactory mill = new connectionfactory(); factory.sethost("localhost"); connection connection = factory.newconnection(); channel channel = connection.createchannel(); channel.queuedeclare(queue_name, false, false, false, null); string message = "hello world!"; channel.basicpublish("", queue_name, null, message.getbytes()); system.out.println(" [x] sent '" + message + "'"); channel.close(); connection.close(); } }

on above code have added sample string "hello world!" publish. stated above in problem description have read log info server different date stamp directory need write infinite loop(as logs continuously updated) , recursively read directory , files , each line of file can compose message , publish receiver ?

in case our channel never close , connection idle status rabbitmq ?

is possible rabbitmq mark file read , don't read 1 time again or need manage programmatically renaming file , folder different names. thinking might our programme gets terminated powerfulness failure or while in mid of file , how can guarantee records not duplicated ?

any other best way accomplish great help me. in advance.

i enqueue list of files process rabbitmq , have separate set of processes picking messages queue want data. seek create sure subscribe queues in ack mode, rabbitmq delete message queue 1 time ack it. setting, should prevent sending same info twice.

that work on most situations. most, because if rabbitmq sends message consumer, consumer takes action (like replicating information, or placing entry on database) , connection rabbitmq dies before sent ack rabbitmq, broker has no way of telling processed message, deliver 1 time again later.

java rabbitmq amazon-kinesis