A 10/10 output is one that required no modifications, minimum iterations, is equivalent to my skills, sufficiently tested, and I don’t have any reservations about future maintenance.

A 0/10 is the AI

Date: 2025-03-19 Model: Claude Sonnet 3.7 Result: 7/10 Prompt:

Update this1 syn 1.0 derive procedural macro to syn 2.0 and make the macro output descriptive error messages instead of panicking.


Date: 2025-04-02 Model: Cursor “auto” agent Result: 0/10 Prompt:

The animation loop tests are failing as time doesn’t appear to be incrementing, can you fix them?

30 minutes of me iterating on the prompt and the agent repeatedly running tests (which is a very cool concept), only to give up after attempting lots of variations of spaghetti code, left a sour taste in my mouth.

The real solution: in order for time in an animation to work, it needs to compute the delta. The tests needed to update time before launching into the animation loop to establish a baseline. It’s a oneline change (for each test). I’m surprised that AI didn’t try this approach in one of its many iterations.


Date: 2025-04-03 Model: Claude Sonnet 3.7 Result: 8/10. The oneshot solution2 appeared to worked flawlessly. Only a couple of nitpicks Context: Chinese supplementary mod for EU43

Can you write a rust function that takes in a byte slice and returns a string, and computes “Escaped Text -> wide char (ucs2) -> UTF 8” conversion. Make it idiomatic Rust

Nitpicks:

  • The output didn’t consume the input idiomatically, preferring to directly index the slice with a variable like shown below:
      let low = input[i];
      i += 1;
      let high = input[i];
      i += 1;
    
    I rewrote it using a combination of [T]::split_first_chunk and [T]::split_first
  • AI was unable to generate meaningful tests. Even though the context didn’t contain any tests, I would have preferred to see tests generated that leveraged Chinese symbols so that I can guard against regressions. So while the output appears to work flawlessly, it could be that edge cases exist.
  • And perhaps due to the lack of tests, my understanding of the domain didn’t increase. I don’t understand how the escaped text encoding works and what underpins its design. At a 10,000 foot view everything is clear, but in an interview, I would fail to explain the inner workings.