Repository navigation
Allow for multiple Topics. #24
Description
Activity
Would this allow for a Topic per message type approach? I was considering forking and implementing this myself.
The reason I ask is purely for throughput reasons. For each subscription to a topic, throughput is reduced. See MSDN:Benchmarks suggest that a single topic with 5 subscriptions can achieve a message throughput of up to 600msg/s >(message size: 1KB) if all messages are routed to all subscriptions. To obtain higher throughput, use multiple >topics.
Benchmarks suggest that a single topic with 250 subscriptions can achieve a message throughput of up to 5msg/s >(message size: 1KB) if all messages are routed to all subscriptions. To obtain higher throughput, use multiple >topics.
In a system with a wide variety of messages and a reasonable number of instances I could see a single Topic approach causing problems.
The way the current design works, I would need to use a non static version of the "bus" class.
My thoughts would be to create a config and pass it to a method to return a sender.
We could overload the send methods and perform a lookup but there would be a slight overhead with that.
How did you plan on implementing it?
I would have some sort of sender factory and have a mapper list (like you do for subscriptions) to route messages to their respective sender. Each sender would have it's own TopicClient instance.
I think the receiver side of things could be accomplished pretty much in config. I'm not that familiar with your code base so I might be way off.For multiple topics, both the sender and receivers would need to support it. My thought on the receiver side is to have a comma separated string so that I can go and register them to multiple topics.
Do you have psydo code that would show how you see this sender factory?
Would it not be simpler to have a separate receiver per message type (with its own mappings)? Maybe make it a generic?
My (vague) plan for the sender factory was to use generics and either reflect the class name of the argument or use a hash as you do with subscriptions and attach a TopicClient (or MessageSender) to each sender.
This has the problem in that the end user would have to request a different sender instance for each message type. It is probably more sensible to maintain a collection of TopicClients inside a non-generic Sender and just reflect on the message type to get the route.Hey Joe please take a look at the latest commit on my fork for my initial implementation:https://github.com/eltone/ProjectExtensions.Azure.ServiceBus
I realize that this forces the multi-topic decision on the user but I felt it was the simplest way to get a working example. I also still need to ensure these changes are in line with your topic/ subscription recovery work.
If you are happy with this direction I can submit a PR and we can go into more detailed discussion over the diffs.
I will try to look at it tonight. It may be a day or so. Thanks for the fork.
I only looked online. Did you make it so that each message is it's own topic? That would mean that the receivers would also need a client per message type.
Yes the CreateSubscription method handles this by fetching a TopicDescription instance from the dictionary using the type name to pass into the AzureReceiverHelper constructor. Currently the sender and receiver use separate sets of TopicClient instances.
I think I misunderstood how you wanted to scale out. What if you want to send 5000 msg / s of one message type. The only want to do that is to spin it across multiple topics. The current code you have would not support that.
I now re-read your original comment so your approach makes sense with the code that you have. What's your thought on the last comment?
I don't think it would be much work to allow for multiple topics per type:
- This could be either configured globally across all types or by using attributes on the messages themselves.
- Receiving is simple as it just involves creating more AzureReceiverHelper instances.
- Sending of messages could be implemented by using a round robin or RNG based load balancer.
We would need to add TopicName to the Configuration data. If it is not configured then it would use the default created when the application was configured.