Benchmark: CadQuery vs. OpenSCAD for agentic CAD work

(modelrift.com)

17 points | by jetter 1 hour ago

6 comments

  • dofm 58 minutes ago
    None of the "findings" here really needed an AI experiment to do it:

    - "CadQuery fails loudly and early... OpenSCAD fails silently and late"

    This should be obvious, from the documentation, from the way geometry construction works. OpenSCAD has no sense of failure where one shape means another won't work; you're just drawing the equivalent of 3D pixels in space. It will always potentially-meaninglessly succeed if the syntax is OK.

    - Renders caught nothing that mattered

    Objects with hollows are not going to show their major problems this way.

    - CadQuery can be interrogated, OpenSCAD cannot

    Isn't this explicit from the documentation of both? One works by iteratively building on top of a previous result that can be stored in a variable; the other doesn't.

    - OpenSCAD renders have no concept of a part edge

    Again — this should be explicit from the documentation, which describes no methods for operating on edges (and largely from the fact that it is declarative).

    - Speed favours OpenSCAD, and it barely matters

    Yes, being faster is no good if things are wrong.

    - "What decides it is verifiability rather than expressiveness, and CadQuery leads there by a wider margin than the syntax difference suggests."

    Yes, because the difference is semantic. Which you can get from the documentation. Building iteratively on the basis of existing geometry is inherently more verifiable, because stuff that can't work won't work.

    Honestly do people not try learning CAD before they try to build an AI tool to generate CAD models? This feels like yet another situation where people who have not done the foundational conceptual work seem to think that they can avoid it and just work around it with AI.

    • ricardobeat 2 minutes ago
      Your mistake is assuming “people” wrote this.
  • fallat 15 minutes ago
    such a garbage post for various reasons, but the ending is the kicker

    after making SEVERAL observations that cadquery is pretty much better overall, they say "ehhh we're going to stay with openscad".

  • ricardobeat 10 minutes ago
    Thanks, Claude!
  • ariwilson 1 hour ago
    not interesting as the writeup as very AI sloppish :(. i'm sad about that as I would really like to have an intuitive sense for how good the current frontier models are at coming up with interesting 3d printable parts.
    • ____tom____ 1 hour ago
      I've been just getting started on this with Astra. Astra is one-shotting good 3D models from photographs for me. I was unable to get Sol to do this, even after multiple rounds.

      It's not a small change, it's from "not useful" to "pretty good"

      • elwebmaster 51 minutes ago
        With any tool?
      • dofm 57 minutes ago
        > Astra is one-shotting good 3D models from photographs for me.

        Right, but, so what? There is very little utility here. Parts that need to be useful need to be designed to spec.

        • ____tom____ 46 minutes ago
          Maybe not for you, but I'm building miniatures for custom board games. Works great.
          • dofm 42 minutes ago
            Right but there is no meaningful specification failure mode in creative CAD, is there? I mean, yes, image to CAD for a little miniature, I can buy that it is OK. Fun, even, especially when your use is sort of transitory.

            But this article is talking about functional parts. A functional part exists because of its spec, usually fails because of some combination of its material, manufacture, and spec. And the spec is chosen with that in mind.

            Even working from a text description of a part is error prone because language is imprecise and interpretable. I just don't understand why AI people want to race past actual proper sensible solutions to a problem just to automate generating from text.

            • elwebmaster 32 minutes ago
              People don't have time (or even mental capability / even more time) to learn CAD but they have a task at hand which they want done. If AI can generate a part which is "good enough" for the user at a cost that the user considers reasonable (I.e. zero) then AI gets used. If a part fails thats rarely a concern because these parts are just 3d printed and likely cost single digit cents in filament. User can always print a new one.
              • dofm 25 minutes ago
                Right, but the thing that I think sort of militates against this scenario is this:

                If a non-CAD person imagines a part that they need, the chances are very high indeed that either it isn't what they need, can't be made robustly or sensibly, or if it is, it already exists in some meaningful way that can be adapted, because one of tens of millions of other people already wanted it.

                This scenario — needing something very specific that can be described only in words and nobody else has ever needed before — is almost imaginary. Most stuff just isn't like that.

                Now… if someone were to design an AI CAD package that could remix objects that exist and combine them, perhaps with a GUI tool that allowed a user to identify attachment points, maybe that would be interesting. But a lot of that has been solved in non-AI GUIs already.

                Text-to-CAD just isn't as good an idea as it seems. You are not going to be able to magic up some novel invention from a text description without a significant amount of domain knowledge of the words needed to describe a part, and that domain knowledge usually comes from learning CAD!

                (Bit like getting a coding agent to write really good code)

                And simple parts, sorry, just don't need much CAD skill. Get a CAD package on a free plan, learn enough to make your thing. It'll probably have standard tools for adding threads, even.

                Even image-to-CAD for a functional part requires a significant amount of understanding of what to photograph and from which angles, etc.

        • Mashimo 51 minutes ago
          Once you have the model, it's relativ easy to change the demensions / constrains later on, no?
          • dofm 49 minutes ago
            Not if the model has not been implemented from reasonable principles.
            • Mashimo 27 minutes ago
              What does that mean in practice?
              • dofm 7 minutes ago
                What it means is that a good CAD model is not just sticking stuff together in 3D space.

                Parametric flow means that a small number of measurements can ripple through the design, and sketches and solids adapt. To do this right, it's easier to just understand the purpose and intent of the model, how it was manufactured, and draw from there.

                One of the things you get from studying CAD diagrams — like the Solidworks exercises everyone works through — is that real world parts often have surprisingly few dimensioned constraints. A lot more emerges from geometry.

                So you can look at a diagram and initially think "where does it define this thickness, it's not in the diagram!?" and find that it never needs to, because that thickness emerges from tangency, parallelism, perpendicularity combined.

                Specifying it would therefore be overconstraint; it might lead to models that break when changed. The diagram will specify the actual driving measurements that come from the part's real world purpose — what it fits into, what it connects to., and with the intent to adapt to different real-world requirements.

                There's a bit in this article that really illuminates why OpenSCAD is rarely the right choice (CadQuery is admittedly better).

                > OpenSCAD needed eight versions to CadQuery’s five, and three of those eight went to boolean hygiene rather than design: cleaning up slivers thrown off by tangent and coincident faces.

                The weird thing in this is that this is not a question of hygiene at all. They are talking about removing fragments of solids that exist because OpenSCAD has no notion of faces, edges or vertexes, and therefore no notion of coincident and tangent faces.

                But tangency is not just appearance and tidiness: it has an impact on part strength (shear strength for example) and manufacturability.

                So this is a question of CAD-aware design. In a CAD package with a sketch-driven workflow, you can tell it explicitly about tangency, because you know it exists.

                Any CAD tool that works from images is going to have to approximate, which will create over-complex models that are more fragile with respect to changes, whereas a design that has geometrical constraints in place that are logical will be simpler and more adaptable.

                I must stress that I am an amateur. But my amateur skills have come from reading and learning about CAD and learning what makes a viable design.

                With functional parts, roughly, form follows function first, and manufacturability second. But you have to understand why, to be able to make a good part. Even (perhaps especially) with 3D printing. But not uniquely. For example, would this tool be able to understand why the draft angles on a plastic part were important, what determines the bend radius of a part that was made from pressed metal, etc.?

                (Sorry… this was a stupidly long answer but I am hungry and I figured I should try to blurt it out before I have to get up from my desk and cook)

    • rjsw 1 hour ago
      There was a blog post from a couple of days ago that had been initially flagged, it seemed to me to have a good overview of AI for CAD, will try to find the link.
    • dofm 36 minutes ago
      [dead]
  • tamimio 44 minutes ago
    Should try different models and harnesses, CC and opus5 aren’t the best combo. Also, I expect soon to have models that verify the mesh and even render it not just generating the code. So we got this for mechanical engineering, cupperhead and similar for electronics, usual agents for software, what is next?
  • brcmthrowaway 50 minutes ago
    The OP seems to be shilling this tool nonstop. Modelrift just seems to be an LLM wrapper.. the original models can handle OpenSCAD just fine.