Skip to content

Coding, art, and the pleasure of bending a medium

A conversation with Minnie raised a question: is coding an artistic skill? I would argue that it is. Let's go over why I feel that way.

Code is already part of my art practice

I work in roughly 30 mediums (somewhat regularly). A lot of them look more like coding than painting.

Two of them that actually involve coding are 3D (with Blender), and some of the musical work I do in my DAW. Both involve Python scripting.

ZBrush has its own language. Unity uses C#, and a lot of the 2D motion graphics work I do involves scripting in JavaScript.

I also do a lot with character building tools (VRoid Studio, CC4, Daz, Poser, and so on) which is all about balancing "morph targets."

In English, what that means is that I'm juggling the right set of deformations, with numbered sliders, that, in concert with each other, ultimately produce the piece of art I'm trying to make... by dragging them up and down.

Everything else in this category is keyframing, curve animation, and various types of mocap, which obviously aren't art either, pshh... just kidding.

This has always been considered a valid artistic skill set, and it is indeed the entire art form.

I've been doing those, on and off, since about 2007 and the release of Poser 4 by MetaCreations. Nobody's ever argued that it's not art, even when people were obsessed with telling me that somehow... Daz represented the end of character art (incidentally, it didn't).

The argument there was that it was "too easy."

Not that it wasn't art.

Tools, exhibitions and sculpture

Two interesting things here, that you can verify, if you want to, are technical schemes I came up with to automate the creation of the ARKit blendshapes in Blender, as well as the material transfer tools I wrote for Unity... with code.

One of my first experiences as a digital artist was back in the QBasic days, moving turtles around, and making it rain. Experimenting with making fractalized trees with recursive goto loops.

That particular exercise, by the way, requires a lot more control than "traditional" digital painting.

I've done entire exhibitions that were procedurally generated GIFs created also entirely with code.

And every bit of this is before I even mention some of the 3D-printed sculpture work I've done, which, as you know, clearly depends on both coding and 3D.

Why AI feels different from coding

Now, with that long preface aside, AI doesn't really feel like coding to me. Here's why:

Coding is kind of rigid. It has rules. It has pre-defined structures like objects, constraints, functions, classes, and so on. The math is very direct and you have probably a lot more control than you actually want as an artist, when you're using it to make art.

Now, if I don't mention it, someone will point out that yes, you can write your own Automatic or comfy modules in Python, and do the thing with code there, and you absolutely can. And I have done both of these things.

And yes, fine-tuning can get very technical.

But I don't think most people do AI that way.

I think there's an even split in the community. Those who prompt raw, and those who prompt against their own drawings or groups of drawings.

In both groups, about half of them are actively trying to break the machine, and subvert the use case in novel and interesting ways. These people are called Neural Benders.

I am one of these people.

Overlap does not make them the same artform

But without too much of a tangent about my excitement around neural bending, I think comparing prompting to coding is like comparing coding to accounting.

Yes, there is overlap.

No, they are not the same artform.

Accounting and actuarial stuff breaks my brain. It's the single most boring thing I can think of to do with numbers. It involves rules related to the law, rules related to factoring and percentages, and velocity. Sometimes, it even involves predictions, and there's a lot of analysis.

Coding by comparison is more creative. You can think of how you want to solve problems, and factor up a solution to get there, with any rules you want, no matter how crazy and incoherent they are. If you've gotten from Point A to Point B without an execution error, you my friend have succeeded as a coder, no matter how good or bad your coding skills are.

Bending the medium

Visual AI uses antiquated language models, like the kind you see in Siri or Google Home to create new artistic compositions, based on some very broad patterns and opinions on art in any given neural network based on text, images, and various types of depth mapping.

One key difference between AI and coding (at least in the Automatic workflow), is that you can't break the system with the prompt you enter into it.

With coding, if you make a syntax error, it'll stop executing. By contrast, an AI breaks everything down into simple arithmetic, and it deducts one from the other. And it keeps going, even when there's a negative outcome.

That's what makes it exciting as an artistic tool.

Once you develop a certain amount of skill doing it, you'll find that you can bend AI in ways that humans have trouble visualizing on their own. It's like collaborating with an alien who understands our entire universe by looking at postcards and memes.

The art happens when you realize that everything you do and say to this totally alien thing influences the machine in ways that cause it to get very confused.

And you're an artist when you do it, because you are doing something to change the universe with your will, and your various types of drawings and image maps.

I don't care what you call it.