Recent Forum Posts
From categories:
page 1123...next »

Dear All,

I skip the part where I say how much I love ZMQ and how great/easier it makes my dev….I guess this is implied in this thread trying to implement it in yet another project…

So, I would like to listen on 3 TCP sockets A, B & C: actually 2 clients (A=25500 & B=27500) and a server (C=25000)
Each A, B or C socket should/will receive string CSV packet, where the third parameter is the destination.
Based on this destination, packet will be redirected/forwarded/enqueued to the appropriated socket for sending.
No ack, no validation, no query-expected-response…
If connexion drop on one of the socket, packets for this one are just dropped.

So there is little check for the queue selection, and simply 3 sockets.

Question:
- what is the best implementation? not sure I am aiming at a PUB/SUB here (or a double one for each socket?), no REQ/REP, nor BROKER…
Actually this _could_ look like the famous weather example, however, not only subscribing in one direction, as each inbound is able to send/receive those string packets.. I guess I don't want FORWARD (although very tempting) since it won't le me do my packet/csv inspection at code level…or will it?

This sound so…easy, yet I could not manage to come with a great (thus simple) design…

Improvement:
- a fourth (or N-th) socket could be added in the futur, design should remain flexible.
- this would be initialy Linux/Python, but I don't have any issue with other language (although I am confident this is not really plateform/language dependent, right?)

Could you come with

alick (guest) 01 Mar 2017 09:57
in discussion Hidden / Per page discussions » ZeroMQ for MySQL

the problem has bean resloved bad code line 43 at zeromq.c,the function strlen argument not be a NULL value

by alick (guest), 01 Mar 2017 09:57
alick (guest) 01 Mar 2017 09:15
in discussion Hidden / Per page discussions » ZeroMQ for MySQL

9:14:16 UTC - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.

key_buffer_size=8388608
read_buffer_size=16777216
max_used_connections=38
max_threads=800
thread_count=37
connection_count=37
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 39340442 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x46d304b0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong…
stack_bottom = 7fdaf8c29d30 thread_stack 0x40000
/usr/sbin/mysqld(my_print_stacktrace+0x3b)[0x91068b]
/usr/sbin/mysqld(handle_fatal_signal+0x491)[0x693c81]
/lib64/libpthread.so.0(+0xf130)[0x7fe5be672130]
/lib64/libc.so.6(+0x162811)[0x7fe5bcf52811]
/usr/lib64/mysql/plugin/libzeromq.so(zmq_client+0xb4)[0x7fe5745f7f05]
/usr/sbin/mysqld(_ZN11udf_handler7val_strEP6StringS1_+0x87)[0x62f537]
/usr/sbin/mysqld(_ZN17Item_func_udf_str7val_strEP6String+0x1c)[0x62f5fc]
/usr/sbin/mysqld(_ZN4Item4sendEP8ProtocolP6String+0x237)[0x5dea67]
/usr/sbin/mysqld(_ZN8Protocol19send_result_set_rowEP4ListI4ItemE+0xc5)[0x68d445]
/usr/sbin/mysqld(_ZN11select_send9send_dataER4ListI4ItemE+0x5f)[0x6e1c8f]
/usr/sbin/mysqld[0x6f5ff0]
/usr/sbin/mysqld[0x6f14ef]
/usr/sbin/mysqld(_Z10sub_selectP4JOINP13st_join_tableb+0x149)[0x6f1899]
/usr/sbin/mysqld(_ZN4JOIN4execEv+0x2e8)[0x6f0a78]
/usr/sbin/mysqld(_Z12mysql_selectP3THDP10TABLE_LISTjR4ListI4ItemEPS4_P10SQL_I_ListI8st_orderESB_S7_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x275)[0x73c6d5]
/usr/sbin/mysqld(_Z13handle_selectP3THDP13select_resultm+0x165)[0x73cf35]
/usr/sbin/mysqld[0x5816e8]
/usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x1cb5)[0x714965]
/usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x3b8)[0x71a9d8]
/usr/sbin/mysqld[0x71b231]
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x1b72)[0x71d5f2]
/usr/sbin/mysqld(_Z10do_commandP3THD+0x27b)[0x71f10b]
/usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x162)[0x6e7622]
/usr/sbin/mysqld(handle_one_connection+0x40)[0x6e77e0]
/usr/sbin/mysqld(pfs_spawn_thread+0x143)[0xb55993]
/lib64/libpthread.so.0(+0x7df5)[0x7fe5be66adf5]
/lib64/libc.so.6(clone+0x6d)[0x7fe5bcee61ad]

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (7fdadc00f640): is an invalid pointer
Connection ID (thread ID): 52
Status: NOT_KILLED

by alick (guest), 01 Mar 2017 09:15
alick (guest) 01 Mar 2017 09:05
in discussion Hidden / Per page discussions » ZeroMQ for MySQL

the zmq_client('socket','message') funciton ,whe message==NULL mysql server will crash, mysql Version=Ver 14.14 Distrib 5.6.28

by alick (guest), 01 Mar 2017 09:05

Number 21, big grey door (it's open), walk into the parking lot and in the opposite corner climb up 4 steps of stairs and you will see a big blue door, a cargo elevator. It's floor 4.

How to get there by Luca BoccassiLuca Boccassi, 02 Feb 2017 08:10
kalki70kalki70 16 Jan 2017 19:03
in discussion Hidden / Per page discussions » ØMQ is just Sockets!

¿You can achieve the same using plain sockets?

Can you take a plain socket, and call bind() for several addresses/ports on it ? Can you take a plain socket and call connect() simultaneously to different destinations? And then, can you call send() and distribute load over all those destinations, using a plain old socket? If those destinations disconnect, your plain socket reconnects automatically? The list here is really very long…

by kalki70kalki70, 16 Jan 2017 19:03

Hello,

I have a problem using ZMQ_PUB. I have a producer which connects to different processes that bind on ZMQ_SUB sockets.
When producer starts it creates s ZMQ_PUB socket and connects it to different processes. It then immediately start sending messages at a regular period.
As expected, if there are no connected subscribers, it drops all messages, until a subscriber starts. The flow works normal then, the subscriber start receives the messages. Now, the problem is :

  • I disconnect the subscriber (stopping the process).
  • There are no active subscriber at this point, as I stopped the only one.
  • The producer continues sending messages, which should be dropped, as there are no connected subscribers anymore…
  • I restart the original subscriber… and it receives all messages produced in the meantime !!

So what I see is that the producer enqueued all messages while the subscriber was down. As soon as the socket reconnected, because the subscriber process restarted, it sends all queued messages.

As I understood from here, a publisher should drop all sent messages when there are no connected subscribers:

As I am a "low-karma" user I can't publish a link to the documentation, which states :

"A publisher has no connected subscribers, then it will simply drop all messages."

Why is this happening?

By the way, I am using C++ over linux for these tests.

Thanks in advance,

Luis

ZMQ_PUB enqueuing messages? by kalki70kalki70, 16 Jan 2017 16:04

The NuGet package manager in visual studio currently offers the ZeroMQ C# binding versioned 4.1.0.21 (for the clr namespace). Installing this package provides the libzmq.dll with a version of 4.1.5. The application I am working on utilizes a binding in another language which is most stable using an older 3.x version of the libzmq.dll. What are my options for the C# binding that will be compatible with a 3.x version of the libzmq.dll?

Can I run the latest binding on the C# side using the 4.1.5 version of libzmq.dll and use the legacy version of zmq from the other component of the application? In general, do all components using zeroMQ to communicate need to use the same libzmq.dll version?

Can I run the latest binding on the C# side with a 3.x version of the libzmq.dll?

Is there a good option for downgrading the binding and libzmq.dll on the C# side, for example the clrzmq package on NuGet?

Gopi (guest) 13 Dec 2016 12:23
in discussion Hidden / Per page discussions » ØMQ is just Sockets!

It is just sockets. I can achieve the same using plain sockets. How it is different from other JMS implementations

by Gopi (guest), 13 Dec 2016 12:23
Kgothatso (guest) 21 Nov 2016 14:03
in discussion Hidden / Per page discussions » NORM Transport Notes

Hi. I built ZeroMQ manually as prescribed above. I point it to the norm source tree path, and on the make steps it shows that it builds the norm_file. However, I still get a "protocol not supported" issue when trying to run my app. Am I missing something? Also, in the Qt Creator .pro file, how do I add the NORM library? (e.g. LIBS += -lzmq for ZeroMQ)

by Kgothatso (guest), 21 Nov 2016 14:03
kumar (guest) 10 Nov 2016 08:58
in discussion Hidden / Per page discussions » High-speed message matching

the article explains this ..
u build a table for all the criteria say sub1 is orange.#.yellow and sub2 is apple.custard.*

0 orange
1 yellow
2 apple
3 custard

so sub1 bitmap will be 1100 and sub2 will be 0011
and then for each message u create a bitmap

msg apple.red.green
0 1 2 3
msg1 0 0 1 0
sub1 1 1 0 0
---
0 0 0 0 ->its not 1100 so message doesnt match, also no matching fields

msg1 0 0 1 0
sub2 0 0 1 1
---
0 0 1 0 ->its not 0011 so message doesnt match, but has one matching fields

msg orange.color.yellow
0 1 2 3
msg1 1 1 0 0
sub1 1 1 0 0
---
1 1 0 0 ->its 1100 so message matches, chk for order though

by kumar (guest), 10 Nov 2016 08:58
Daniel Cadenas (guest) 14 Oct 2016 22:37
in discussion Hidden / Per page discussions » Broker vs. Brokerless

Ankush notice in the pseudocode that each function returns the value of the inner function to the caller. That's what those arrows represent. We are popping a stack.

by Daniel Cadenas (guest), 14 Oct 2016 22:37

Hi,

I am using a GUI (client) with req socket in it and beaglebone (server)with rep socket. I use lazy pirate pattern in request reply format.
I am able to commnicate with one client and one server.
When I use multiple GUI(clients) to a single beaglebone(server) there are communication breakups after few hours of running.
How to handle multiple client request at the server side?

Regards
jo

Multiple clients by jojitjojit, 24 Aug 2016 13:02

Hi there,

Working on a proof of concept on reading a queue from zeromq (pub/sub) and displaying data on a browser using node,socket,jquery.

Its a monitoring application running from crontab that starts every 5 minutes and feed arbitrary data to a zeromq queue.

My problem is how to signalize the end of that "iteraction" or the start of the next one (5 minutes period)?

I am using

<script> var skt = io(); skt.on('event', (x) => $('#msg').append($('<li>').text(x.fail))); </script>

and it gives me an appending list.

My intention is only have the latest iteration. Any way to mark the last check?

regards,

Hi, I'm new to the idea of distributed systems, so please pardon if the question appears silly. In the first diagram I don't understand the existence of arrows #9 to #12. From the function pipeline, it'd appear that the processing finishes when App D is done. As per me, after #8, we have the output, which should take at most two more network hops.

I suppose this has to do something with SOA architecture, but I'll be happy if someone could explain.

Help explain diagram 1 by Ankush ThakurAnkush Thakur, 20 Aug 2016 13:07

Forget it guys, I don't think he's around…

Re: Small correction by rohitggargrohitggarg, 31 May 2016 09:21

pull request => s33.postimg.org fcstyp8v3/broker2.png

Re: Small correction by rohitggargrohitggarg, 25 May 2016 09:20
Jason Baik (guest) 03 May 2016 15:36
in discussion Hidden / Per page discussions » Measuring messaging performance

Wow, this article is an absolute jewel. I am doing a performance test with ActiveMQ, and this has helped me alot so far. Thanks!

by Jason Baik (guest), 03 May 2016 15:36
Ivan (guest) 10 Apr 2016 15:46
in discussion Hidden / Per page discussions » NORM Transport Notes

Hey, you need to build zeromq manually with ./configure —with-norm=<norm source tree path>

by Ivan (guest), 10 Apr 2016 15:46
pratik inamdar (guest) 26 Mar 2016 20:44
in discussion Hidden / Per page discussions » Traffic Monitoring

I am not able to execute the above monosender.c file itself. Please help.I am getting error /usr/include/zmq.hpp:30:21: fatal error: algorithm: No such file or directory
#include <algorithm>
^

Please help

by pratik inamdar (guest), 26 Mar 2016 20:44
page 1123...next »