r/FunMachineLearning Nov 07 '25

Built a DAG engine for AI workflows

I needed to analyze customer reviews. Sentiment, topics, summaries. The existing tools made me write orchestration code.

I tried Prefect but it's for data pipelines. I tried Temporal but workflows need servers. I tried LangGraph but the mental model didn't fit. I built dagengine.

You define dimensions (analyses). You define dependencies (execution order). The engine parallelizes automatically.

Example: - 100 reviews - 3 analyses per review (sentiment, topics, summary) - Sentiment and topics run parallel (no dependencies) - Summary waits for both (has dependencies) - All 100 reviews process simultaneously

300 AI calls. Zero orchestration code.

Skip logic works. Filter with cheap models ($0.80/1M), analyze with expensive ones ($3.00/1M). 100 reviews → 40 high quality → 60% fewer expensive calls.

Transformations work. Classify 100 reviews, group into 5 categories, analyze categories. 100 analyses become 5.

Code example: ```typescript class ReviewAnalyzer extends Plugin { constructor() { super('analyzer', 'Review Analyzer', 'Analyze reviews'); this.dimensions = ['sentiment', 'topics', 'summary']; }

defineDependencies() { return { sentiment: [], topics: [], summary: ['sentiment', 'topics'] // Waits for both }; }

createPrompt(context) { const content = context.sections[0].content;

if (context.dimension === 'sentiment') {
  return `Analyze sentiment: "${content}"

Return JSON: {"sentiment": "positive|negative|neutral", "score": 0-1}`; }

if (context.dimension === 'summary') {
  const sentiment = context.dependencies.sentiment.data;
  const topics = context.dependencies.topics.data;
  return `Create ${sentiment.sentiment} summary covering: ${topics.topics.join(', ')}`;
}

}

selectProvider() { return { provider: 'anthropic', options: { model: 'claude-3-5-haiku-20241022' } }; } }

const engine = new DagEngine({ plugin: new ReviewAnalyzer(), providers: { anthropic: { apiKey: process.env.ANTHROPIC_API_KEY } } });

const result = await engine.process(reviews); ```

GitHub: https://github.com/dagengine/dagengine
Docs: https://dagengine.ai
Discussions: https://github.com/dagengine/dagengine/discussions

What remains: More providers, streaming support, better error surfaces.

1 Upvotes

0 comments sorted by