作者:carefulff | 来源:互联网 | 2023-07-15 20:16
ImworkingwithaSoftwareOrientedArchitecturethathasmultipleceleryworkers(letscallthem
I'm working with a Software Oriented Architecture that has multiple celery workers (let's call them worker1
, worker2
, and worker3
). All three workers are separate entities (i.e., separate code bases, separate repos, separate celery instances, separate machines) and none of them are connected to a Django app.
我正在使用一个面向软件的架构,它有多个芹菜工人(我们称之为worker1,worker2和worker3)。所有三个工作者都是独立的实体(即,单独的代码库,单独的存储库,单独的芹菜实例,单独的机器),并且它们都没有连接到Django应用程序。
Communicating with each of these three workers is a Django based, MySQL backed RESTful API.
与这三个工作者中的每一个进行通信是基于Django的MySQL支持的RESTful API。
In development, these services are all on a vagrant box, each acting as a separate machine running off of a separate port. We have a single RabbitMQ broker for all of the Celery tasks.
在开发过程中,这些服务都在一个流浪盒上,每个服务器都作为一个单独的端口运行的独立机器。我们为所有Celery任务提供了一个RabbitMQ代理。
A typical path through these services might look something like this: worker1
gets a message from a device, does some processing, queues up a task on worker2
, which does further processing and makes a POST to the API
, which writes to the MySQL DB and triggers a task on worker3
, which does some other processing and makes another POST to the API
which results in a MySQL write.
通过这些服务的典型路径可能如下所示:worker1从设备获取消息,进行一些处理,在worker2上排队任务,进行进一步处理并对API进行POST,写入MySQL DB和在worker3上触发一个任务,它执行一些其他处理并对API进行另一个POST,从而导致MySQL写入。
The services are communicating nicely, but it's very annoying to test this flow every time we make a change to any service. I really want to get some full integration tests (i.e., starting at a message sent to worker1
and going through the entire chain) in place but I'm not sure where to start. The main problems I'm facing are these:
服务很好地沟通,但每次我们对任何服务进行更改时测试此流程都非常烦人。我真的想要进行一些完整的集成测试(即从发送给worker1并通过整个链的消息开始),但我不知道从哪里开始。我面临的主要问题是:
If I queue up something on worker1
, how can I possibly tell when the whole flow is over? How can I make reasonable assertions about results when I don't know if the results have even arrived?
如果我在worker1上排队,我怎么能分辨出整个流程何时结束?当我不知道结果是否已经到达时,如何对结果做出合理的断言呢?
How do I deal with DB set up/tear down? I want to delete all of the entries made during a test at the end of each test, but if I'm starting the test from outside of the Django app, I'm not sure how to efficiently clear it out. Manually deleting it and recreating it after every test seems like it might be too much overhead.
如何处理数据库设置/拆除?我想在每次测试结束时删除测试期间所有条目,但如果我从Django应用程序外部开始测试,我不确定如何有效地清除它。手动删除它并在每次测试后重新创建它似乎可能是太多的开销。
2 个解决方案