Domain Driven Design in Apache Spark?

Vote count: 0

This is duplication of the thread ([Apache Spark and Domain Driven Design][1])

However the question did not receive any valid answer so I wanted to start the thread again.

There is a comment that says - "Spark is intended to be used with simple, but memory/computation demanding jobs. While DDD should be more of a complex consumer of the results of the Spark job(s)." However as the OP of the thread points out - Eventually these systems that use Spark will become complex and will need to be maintained.

Many a times in the work I do I end up putting a lot of business rules in the spark jobs. However this means that those are not converted into entities or value objects or services, and I end up without any ubiquitous language. I basically end up implementing everything with a some variant of join conditions. And even though Spark is (unit?) testable, and I can document what I'm trying to do through the tests, I still feel I am missing something.

Apologies for having the abstract question, please help me improve this if you feel its required.

asked 14 secs ago

Dhruv Kapur

Let's block ads! (Why?)

Domain Driven Design in Apache Spark?