Transcript#
This transcript was generated automatically and may contain errors.
Welcome to The Test Set. Here we talk with some of the brightest thinkers and tinkerers in statistical analysis, scientific computing, and machine learning. Digging into what makes them tick, plus the insights, experiments, and OMG moments that shape the field.
In this episode, we sit down with Alex Hillman, author of The Tiny MBA and owner of the co-working space Indie Hall in Philadelphia. And I'm in Philadelphia, I go to Indie Hall, I'm just a really big Alex fan. We talk to him a bit about marketing to open source communities and how that's a healthy thing, and also about his use of AI as part of a lot of his community building work. So with that, super excited for this episode, Alex Hillman.
All right. So Alex, welcome to The Test Set. So honored to have you.
Glad to be here. Thank you for the invitation.
Yeah, no, no, I'm pleased as punch. And I think just by way of introduction, so you're Alex Hillman. Importantly, we're both from Philadelphia. You're a co-founder at Indie Hall, which is, as I understand it, one of the first co-working spaces in America?
That's right. We were the first in Philadelphia, one of the first in the world.
Word. That's incredible. You co-founded Stacking the Bricks, which includes the book Tiny MBA, and also more recently, you've really been cooking on a personal assistant that you called JFDI, or Just Frickin' Do It, which I'm so excited to get into a talk about. I'm just going to introduce co-hosts really fast. So joined by Hadley Wickham, who's chief scientist at Posit, and Wes McKinney, who's a principal architect at Posit, and has really been cooking on AI tools as well. So excited to have everyone here.
I think everybody's been cooking on AI these days, so I'm excited for the conversation since I've been following some of Alex's work from afar and have done my own kind of mini assistant work. Certainly nothing to the scale of what Alex has done or what people in the open source community have been doing, but definitely an interest area like personal productivity with AI.
Co-working, open source, and community building
Yeah, same. I feel like what's so interesting about you, and maybe for context, I'm a member of your co-working space. I always find it so interesting all your community organizing and wrangling, but I thought maybe like a fun question for everyone, and especially like Hadley and Wes, to just kick it off is like, have you all read books like the Tiny MBA? Because I find open sources interesting where you're also kind of like putting a tool in people's hands and trying to get it out there.
I mean, I've read some books on like company culture and startups and things like that. So like, you know, the lean startup and I like Ray Dalio's principles. Ray Dalio is a little bit of a divisive figure, but actually it's hard to apply all of his principles. Like it's a little bit hardcore, but I feel like some of the basics like radical transparency and people like being honest and saying what they mean and being willing to like give feedback. And it's pretty interesting. Another book that I like that's a little bit of an MBA book is a book called The Culture Map, because I've worked in like very multicultural companies and teams, and it helped me realize that like the way that I approach work and business is very American.
And in part because we were early, also in part because my background was in open source, my philosophy was like, well, it wasn't my idea. I borrowed from a bunch of stuff. Let me share everything that I've learned and put it back out there. Created sort of like an open source, but not for software cycle that put us on the map internationally, even though we were only one location in Philadelphia and we've worked with, I've worked with people creating independent co-working spaces on every continent except for Antarctica.
But the interesting and completely unexpected thing for me was how that became a gateway into exactly what you were just describing, Wes, which is, you know, co-working is sort of this microcosm for work culture. And I'd argue in its best state, it's a microcosm and a laboratory. It's not just what work is, it's what work can be.
When you kind of play with positional power, I don't want to say remove it entirely, but you get a sort of a different scope of it. And when I got to work with other people creating similar work environments where a lot of the hierarchy, not even hierarchy, just where the infrastructure was either different or removed, I kind of got to see a part of the way I think about it is, you know, you do have that American, specifically North American, specifically U.S. based hyper-transactional version of work and business, sort of at the core of a lot of the culture.
And at the most polar opposite end of the spectrum, a lot of Asian cultures being more collectivist at their core and how that shows up in business and work relationships and communication and all those kinds of things. And even just like having to translate some of the workshops and realize that things I would have to teach Americans was actually preceded by having to help them unlearn certain things. And then I would go run the same workshop in Indonesia and I'd say a thing and they'd be like, yeah, yeah, we know that's that's that's normal here.
Some of the most fascinating work I've gotten to do is the furthest from home, but also in this weird way was where I felt most comfortable. Like I've always felt like kind of a weirdo in the American business culture because I care so much and almost hyper-prioritize relationships in the other direction and feel like a bit of an outsider in that way.
Marketing for people allergic to marketing
Yeah, it's super interesting to hear. And one thing I noticed, too, was that people like Julia Evans had taken your workshops like 30 by 500 and mentions like, oh, what I learned about marketing. I learned from Alex Hillman. And I'm just curious, like what your kind of pitches to people like that or what how you kind of explain the need to to people like that.
Yeah, I mean, the audience for Stacking the Bricks, depending on who I'm describing it to, I would say it is marketing and sales for people who are allergic to marketing and sales. And there's a pretty high overlap between that and people who are traditionally engineers, programmers, artists, typically very good at what they do and pretty bad at selling it. And the way that we've framed it, that I think it's landed with folks like Julia and many of the thousands of students we've worked with, is we very intentionally call out why people are often allergic to these things.
And it's because their experience with marketing and sales is often like deception and manipulation and things that would erode trust. And our entire approach and philosophy through everything we've done from the start is it's about building and more specifically earning trust first and then reinforcing that trust over time. And then what that trust allows you to do in terms of making offers and communication.
You know, again, even when we are doing a sales pitch, the goal is not to persuade you to do something you weren't otherwise going to do. You know, there are manipulative people in this world, of course, but I always tell our students, if you're worried about it, you're not going to do it. You're not that good at it. I promise. It's going to take a lot of work to get so good at manipulation that you do it by accident. But if you think about persuasion less about getting somebody to do something that they weren't going to do and instead helping them make a decision that is in their best interest, and if you do things right, is in your mutual best interest, then good news. That's sales. That's marketing. That's really all it is at the end of the day.
And the only other thing I'd add is I think what I just described is more the sales end of the spectrum. I think people think of marketing as broadcast. It's I got to talk, talk, talk, talk, talk, talk, talk. And that's a part of it. But I think the part that people overlook because they're always on the receiving end of somebody else talking is how much of marketing is actually listening. And at the heart of the course that we teach, all of our material, my book is really a bunch of different systematic and structured approaches to knowing what to listen for and to do like a mix of pattern recognition and deductive reasoning from high volume listening.
At the heart of the course that we teach, all of our material, my book is really a bunch of different systematic and structured approaches to knowing what to listen for and to do like a mix of pattern recognition and deductive reasoning from high volume listening.
And people don't think about they think about market research, but they think of that as like surveys and focus groups. And we take a totally different approach that I think is really important. I think it both resonates with and plays to the strengths of the systems minded software engineer, the perhaps introverted or shy communication skills that that or just personality traits that people are coming to it.
If I'm somebody who does not consider myself a boisterous extrovert, how am I supposed to compete in a noisy landscape? Might not be a thing people say out loud, but it's certainly a thing that people think. And I think that part of the reason our approach has resonated so much is because it acknowledges that and says, not here to change you. If anything, I want to play it to the strength that you do have and help you see them as a lever towards these particular outcomes, which, again, is mutual gain between you and your customer. And the way you get there is by earning and keeping their trust.
Spending and investing trust
How do you think about like sort of spending trust, right? Like it's definitely something you can kind of say, like, I have accumulated all of this trust in the community and now I'm going to like deliberately like I'm going to do something deliberately that I know is going to erode trust, but like I believe it's worth it. Like, how do you kind of think about that?
That's a really interesting question. So, I mean, part of it is I try not to erode trust or more specifically of what I'm doing is going to erode trust. I'm going to kind of flip your question around and I'll ask the question, what are the things that typically erode trust? And I think it's one of two things. It's either I feel like you're trying to pull one over on me, like being deceitful in some way. And again, I think most people have experienced that and don't want to do it. But if you don't want to do it, the odds of you doing it by accident round to zero.
I think the more common thing is is that people worry about the message or the request or whatever it is being for everyone, and they end up with a pretty generic and fairly ineffective in all ways message. Right. And instead, part of the philosophy that I approach it with is to think of it almost like cycling through the wheel on Price is Right, stay with me, where you've got a bunch of slices of a wheel and that wheel is your audience, and I try to speak to all of them at once.
I'm going to have to say some pretty hokey things in order for it to be believable for all of them, and it's kind of kind of collapsing on itself. And so instead, it's like, hey, what I'm about to offer isn't necessarily for everybody. But if it's for you, it's extremely for you. And if it's not for you, my hope is that you stick around because the next time I come around, I want to have something that's for you.
I think the key piece is to be aware of what would actually erode that trust and go, do I actually have to erode trust in order to do this? And if I do, sometimes just acknowledging like, hey, if this isn't for you, I hope you stick around because, you know, the stuff you are here for is is is not going away.
I guess one of the things I kind of think about is asking people to do things that are like a little like painful or annoying, but like I 100 percent in my heart of hearts believe you're going to be better off for doing this thing or like, you know, learning this thing that's annoying or you're going to go and change some config, you know, it's annoying. But in the end of the day, you're going to be better off. Like the question I ask myself is how confident am I that the vast majority of them are on the other side of it going to say that kind of sucked, but it was worth it?
If I'm not sure, I'll look for ways to kind of de-risk that or or make it clear who it is more right for. But also, you know, I think coming at this through the lens of an educator and we specifically ask students to kind of trust the process, I think that the thing that has made the biggest difference is the longer the longer we wait for there to be a feedback loop for them, that something is working in the right direction, the more willing they are to continue trusting the process.
So I think if this is going to ask for a spend, I want there to be a pretty quick feedback loop for them to know, OK, that spend was worth it. Right. Or it's not. And that's OK, too. And I can bail. But the longer you kind of drag somebody along for them to be like, man, I wasted my time. Um, I think that's if I think about places where people spend trust, it's usually in stringing people along so long that by the time they make the offer, they go, well, if you had told me that up front, I wouldn't have hung out the whole time.
I'll also like personally, I don't mind overinvesting up front where it's like I'm at the point where I want to have so much trust that by the time I ask you to do something that's a little frictioning that you're like, well, you haven't taken me too far off the rails before, so I'm on board. And I feel like, again, there's a lot of different ways to get there. And if I'm totally honest, I think maybe I overdo it sometimes, but I feel like that puts me in a position where the bank, you know, the trust bank accounts to sort of torture this metaphor is is well invested where that might feel like a big spend in a moment. But if I zoom out to the big picture and go, I've actually earned quite a bit more than I'm asking for.
And also the reminder that like so long as that cycle continues long enough, it is like in like earning interest in investing it compounds. Right. So, you know, our case, you know, if I'm want to put this in marketing and sales terms again, like sometimes if we're running a product launch where we're asking folks to, you know, read a bunch of emails to figure out if it is the right thing for them or watch these videos or whatever it is.
It is pretty rare for our students to see our course go through sort of our initiation, onboarding and launch stuff the first time and buy, not because they don't think it's for them, but because they are not ready to buy yet. The most common thing we hear from people who join our program is they're like, I've been reading your stuff for two years, which means to me I need to have two years worth of stuff lined up for somebody to experience and earn for me to earn the credibility and trust that when the time comes for the big ask that the ratio and proportion is so in my favor that I don't even have to ask myself the question of, am I asking for too much?
Building the JFDI executive assistant
So we zoom out a little bit and then kind of make our way to that specific question because I think it is relevant. So, um, uh, first I haven't said it out loud, maybe it's been implied. I have a software development background, but I've never been a programmer professionally. Um, I've done some like front end development, HTML, CSS, JavaScript. Um, I did that professionally, but I always needed software engineers in my life.
Um, to implement anything beyond my own core skillset and, uh, between or like early use of tools like Cursor and then Claude Code, they initially kind of showed up as an extension of that experience where before I could kind of architect stuff because I'd spent so much time working with really talented developers to know enough of not just the language and jargon, but actually like systems. I learned systems design through actual like scale project implementations across lots of industries over my career, but I always needed another person to actually scaffold it out.
Um, even if the, as the tools got better, I was like, this is not a great use of my, my time, my, my skillset, the ramp up for learning. I just couldn't get over it. And I always felt like anytime I wanted to build something or get, get a thing out of my head and onto a screen that was actually like functional systems software. Um, I felt like I was working with 11 mitts on, I would need somebody else to implement it, or I'm just kind of fumbling around with, with two giant thumbs.
Um, and so my, my, and well, my initial entry into like AI software engineering was this again, sort of weird sideways in, uh, the other thing is the fact that like my goal was not to build software. My goal was to simplify and improve parts of my life and my workflow. Um, as a business owner and as a community builder. And so communication is a huge piece of that.
And the, one of the earliest, you know, besides hooking it up to my inbox and being like, can I teach this thing? How do you run my inbox? I think the most common thing people jump to is can I get it to respond to emails for me? And I was very early, a hard, no, I don't even want that. I don't want to respond to emails. I don't even really want it drafting emails. Um, that was not my goal. I'm pretty good at that. It's all of the stuff that gets in the way of doing that.
And so did a bunch of things in my inbox to have it, um, learn what belongs in my inbox, what goes straight to labels. Basically I had a pretty robust Gmail filter system that I had it download, refactor, reupload, test, and then use everything that it learns to do. So like my email filtering is actually almost entirely deterministic, but it is the deterministic system is managed by Claude Code.
Building a relationship manager from 20 years of email
The second system that I built for my executive assistant was a relationship manager tool, because that is the through line of all of my work. I don't think of anybody as a customer or a client. Um, within Indy Hall there, when people refer to them as customers, I'll always correct them to the members. Even if there's a transaction, the relationship prioritizes above that. And so my thought was how can, if this system is successful, how can it help me show up better for the people in relationships that I have in the ways that I struggle to, which leads me to the fact that I hate every CRM that exists on, on, on planet internet.
Um, cause I think CRMs are designed for a sales pipeline to get you from lead to purchased or qualified somewhere along the way, and I just don't think of people that way. But I do want help noticing when, you know, a relationship that was warm is maybe going a little bit cold. I don't need it to do a thing. I don't even really need it to judge, but it's like, Hey, is there a Delta in the pattern here? Is this accelerating or decelerating? And just flag that for me and let me make the decision or if you can infer other things.
So the starting point was, how do I start, how do I build a dossier of the relationships that I care about? And I've tried a bunch of tools. I think the app that came closest for me was clay. Um, which I know has evolved a bunch, even since I used it, but even it had the same problem that I did, which was the first thing it does is ask you to like import all your contacts from your phone. I'm like, well, I don't want to import all of my contacts from my phone. I want to import the people that actually care about.
And so the intersection of these things was how do I figure out who these people are and I'm now teaching at my inbox. I realized if I send you an email, that's a pretty good clue that I care about you in some capacity. Receiving in inbound mail is a useless signal in basically every way. But if I send an email, that's a pretty good clue. And so I basically taught the system to use that as a hook. And then I'll have a job that runs and it looks at over the last period, all of my sent emails.
It intelligently looks at participation. If there's like, if it's multiple people on a thread, it's going to filter out people that didn't actually participate in that thread, you know, they just so happened to get an email from me, but if I'm sending an email or responding to somebody, that's now the hook. And then it uses their name and email address to search forward and backwards in my inbox and build a, a context file effectively for what it can figure out about our relationship.
And they all start the same way with basic contact information, some meta around how, and when we met, were there people who introduced us? Do we work together on certain projects? Um, have we done business together or not? Do we have overlapping friend groups, um, and things like that. And then, then the robot kind of gets to freestyle. After that front matter.
And so depending on the nature of the relationship, this dossier can go anywhere from 30 lines of text to I've got people like, you know, Michael, our mutual friend, Chris Alfano, and I've been friends for like 17 years. I had to do a deep research. Basically I built, I built the, I built it as a sub-agent. And so there's a few different flags for depth and how to do the deepest research it could. And it pulled up stuff that I completely forgot Chris and I had worked on together. And like, there was so many like moments and memories.
So it was this cool, like trip down memory lane, but where that has come in handy is now when I'm interacting with my executive assistant on anything else in the system, that could be something as simple as I found a cool link on Twitter. And I, I want to bookmark this for later. When I drop it into the system, it'll automatically scan the entire system for related and relevant projects that we're working on that it might be useful for. But it also scans my people database. And it goes, Hey, that thing we just filed, here's three people that might find that interesting. Do you want to send it to them? And it doesn't do the sending, but it is bringing that stuff to the surface. And I don't know, for me, sharing links is definitely a love language.
And I don't think the person on the receiving end cares that I had a little bit of like remembering support to be like, Oh, this person would actually really like that. And then I still manually go and I text or email the link, whatever it is. But using, using the depth of and the scale, I've been using the same email address for almost 20 years, being able to build these really robust profiles that still are, um, you know, it's, it's, it's not pulling in like sensitive project detail and stuff like that. It's like, I can look at any profile and I'd feel comfortable showing it to the person.
That was also like a, like a, like a values and ethics guardrail is, is this going to build a profile that if, if somebody found out I had this profile of them, they would be freaked out. Um, and so like, I think while it is, while it is deep, it is predominantly based on my communication. And so it's not like building a profile of their personality, their words, all of those kinds of things. Um, but instead what our shared relationship is, uh, has been, been super powerful.
That was also like a, like a, like a values and ethics guardrail is, is this going to build a profile that if, if somebody found out I had this profile of them, they would be freaked out.
I mean, for me, it, it, it like, I have, I'm like endlessly frustrated with, with like Gmail as a, as a product. Cause I feel like you don't even, you don't even need AI to make a better analytical, like called an analytical experience for email. Like just like, just to be able to have like a condensed timeline of your correspondence with a single person and the ability to like look at the email attachments that you exchange with that person would be tremendously useful and does not require, does not require any AI.
Like I was just looking in my, in my own message vault and I have something like 200,000 email attachments that have been, you know, sent and received. And I think some of those are like tracking pixels and stuff like that. Um, but, but like, you know, my life and like my, a lot of the, a lot of the — since like I've had many professional relationships and a lot of business relationships and a lot of those relationships come down to like exchanging documents back and forth.
Like anytime I invest in a company, like there's a whole bunch of PDF documents that get exchanged and like just like the burden of going through and like finding the right PDF document. Like where is that one PDF document that I was sent? And it's always buried like somewhere in the middle of a thread, which Gmail makes even harder to find. And if you try to search for it in Gmail, like, you know, there's some kind of Gmail ninjas that know the right like search incantation to find, you know, the right file type and they know exactly what to type in the Gmail search box. But like after 20 years, like I still haven't learned that magic syntax, which means that it's like not a good —
Yeah. And so, so that was like, I had definitely like a, like a, like a groundswell of frustration that led me to be like, like, let me get all of my email out of Gmail and like create a system where I can like, you know, essentially synthesize and reconstruct like these relationships, get all the email attachments out. And so now it's like wonderful to be able to ask Claude Desktop, like, hey, could you find this PDF from 2018 in a matter of seconds, like it will pull up the document for me. And so it's just like, Google, like, couldn't you have done that?
Data sovereignty and the rise of local software
I mean, I'm, I'm really excited about the local and private, you know, applications and like the, you know, because a lot of people are talking about the death of software as a service, but actually like, it's not, not that people are going to build new, like replacements for software as a service, but actually like software as a services, like software as a service as a delivery model. It may be like, may be dying, like actually like, like probably for most businesses, like most applications, business applications can just be a SQLite database. Like you don't even, may not even need Postgres.
It's like how many, you know, maybe if there's just like one, you know, one writer to a SQLite database, like holding a mutex, you know, maybe that's like, maybe that's enough. But for a lot of like business productivity applications, especially like applications that serve individuals, like, you know, there's a lot of products that I use that have, where I don't collaborate with anyone, like only I'm, I'm the only user, like I don't share anything with anyone. Like there's no reason that that can't be like an application with a SQLite database and that, and so like, you know, I've probably built in the last six months, like a dozen projects, like material projects that, that are like basically a productivity application with the SQLite database. And they, they work perfectly fine.
Like they run on, you know, a Raspberry Pi in my closet, on my tail scale network. Like I can connect to it from my phone, you know, from anywhere in the world and it runs just fine, you know, and you know, SQLite's powering the messages, like the chat applications on your phone, like iMessage has, has SQLite inside of it, Apple Contacts, like I'm sure all the Google versions of these things are all based on SQLite.
So it's, it's, it's pretty interesting, kind of that, you know, that, that trend towards that type of like local private, you know, thing, and you can just build these applications really fast now with AI. But I'm also like, I don't know, like, I mean, people have one kind of maybe this is a little bit of a hot take. And I'm curious what you think. But I, there, everyone is building like tremendous amounts of software generating tremendous amounts of code with, with Claude Code these days. And yet, like, there actually aren't that many new things that I'm using.
And maybe I can't tell whether it's like, I'm not like not finding things that are useful or like, maybe I'm just like lack of curiosity. But like, you know, I've been putting, I've been building stuff for myself. And then if I think something is useful, I'm putting it out there into the into the world for people to use that they find it useful, then that's great. I think message vault is something that clearly resonated and a lot of people find useful. But I'm, you know, people are, you know, people are singing, you know, praises for like how productive Claude Code is making them. And yet, like, you know, there's a ton of AI slop on GitHub and like, open source projects are just drowning in slop and like a lot of open source maintainers are like disabling pull requests, like disabling issues, because they just can't anymore.
And they're like, you know, it's it's become like very, like, you know, very Wild West and in a sense, and you know, maybe the more optimistic view is that eventually things will solidify and then you know, there will emerge like a wave of new, like, high quality software. So maybe it just takes like somebody like yourself is just like really focused and passionate about making something that's like, really, really good for yourself. And, you know, whether it's, you know, for other people, or if it's just for you, like doesn't really matter as long as it makes you more effective, it makes you more productive.
I mean, because I mean, I'm with you 100%. I think there's a lot of like productivity performance, especially on social media. And I like I write down a large percentage of and a lot of people get met with a mute button for it.
Now, I'll also say I have this kind of working theory that while there is a clear delineation between like good code and bad code, I think just everybody hates everybody else's code regardless of who makes it. And I think there's there's there's a unspoken thing happening where people are like, my tolerance for other people's software is so low that I'd rather have a shitty version that I made with the help of my robot than deal with somebody else's bad mistake. So I think it's about like the locus of control more than it is about the quality of the product in this very moment.
I also think that we are I mean, to your point about the Wild West, it is both the most exciting and the most frustrating part of the whole thing, because it reminds me a lot of early Internet. You know, for me, I'm thinking back to like 2002 to 2009, where people like big companies like Yahoo were building things like Yahoo Pipes as the precursor to Zapier and basically encouraging you to smush websites together that did not give you permission to smush them together or because they had an RSS feed, they effectively did give you permission to smush them together. And then we kind of drifted away from that. And I feel like to a degree, we're drifting back to it.
The difference is, is we've spent the last decade or so operating in a more like iOS version of the Internet where things that people have more people are using the Internet. I think that's a good thing. Most of those people have no idea how the Internet delivers or does what they're doing. And to a degree, I don't think they should have to in order for it to allow them to accomplish those goals. But I think the downside is, is we've now got a bunch of people who don't know how any of this stuff works and to a degree, don't believe they have to care. And maybe they don't, but they have actually the same power that you and I do. And that creates a lot of really interesting friction and tension, both at the cultural level, as well as the, hey, like what you're doing actually does have effects on not just the industry, but the people that are building and maintaining the tools that you don't even realize that you're using.
I think this is especially true of the open source world. I think people, a lot of the people that are new to it and we can just like even just zooming in on the world of open source and like I simultaneously have published more open source software in the last four months than I have in the last 14 years. And that's some of the most exciting experiences that I've had on the Internet in a really long time. And I love that. And I want that for other people. And I had the early experiences like you were saying before, Wes, is like you put something out there and it becomes even moderately popular or successful. You end up with a pile of slop PRs from people who are even less experienced than me thinking that they're trying to be helpful and making more work than it's worth.
And that just feels like a messy thing that I don't know if it's infrastructure, cultural education and values, or my belief is it'll end up being some combination of the two. I think we were dealing with cultural norms colliding, perhaps, is people who are used to the lockdown of iOS style software and even SaaS to a degree where like it doesn't work the way I want it to and I'm out of options until they make a change to now I can make it do anything that I want, even if it kind of sucks and breaks all the time. And that's what people at least think they want in a short term.
I don't know, maybe the big question that I would riff on your question with, Wes, is how many people that are building those tools and talking about it are still using them two weeks, four weeks, eight weeks, 12 weeks later, let alone a year later. And I'd be willing to bet the vast majority of them went back to old stuff because they were building images in their head that showed up on a screen that is very different from a workflow that is integrated into your work, your life, and actually generates more value than it costs to put effort into.
I mean, I think that there's there's a tremendous amount of like, you know, vibe coded, effectively abandoned where where maybe you have a project that is like on the order of, you know, 100 prompts or less. And it got you to like a prototype of something that looks interesting, but maybe if like you took a healthy amount of time and exercise like good judgment and taste and really like dug in on the details, like the design and the architecture and the project scope, and you cut out the stuff that's not needed and you build like a really you can build something that's really useful and compelling.
But now like when I look at a new piece of software on GitHub, like I actually look at the number of commits in the project to try to judge like how much how much time did the person spend on this. And I think based on the number of like, you know, I roughly look at like a like a commit like I assume that roughly one commit and it's like one prompt and like sometimes people will have multiple prompts and that will generate a commit but you know, on the order of that and so like, you can look at the number of, you know, commits slash prompts and get a sense of like, did they spend three hours on this? Did they spend 10 hours or 100 hours and so something where, you know, and again, like and is it is it somebody who's built good software projects in the past so based on like, how much time did they spend on it?
And is it somebody who has like a track record of building things that are good, like actual good software, especially pre AI, I think are become pretty strong indications of whether like you're looking at something that is, that is useful. But but I even I found it really interesting. Like I started releasing, you know, AI software projects that I built with AI, I think starting in last October, the first one that I put out there where I was like, full disclosure, like I made this with Claude Code, like, don't judge me, like I didn't write the code Claude Code did, but I wrote the prompts.
And and I, I found it interesting, because there were a lot of people that that began to regard the things that I was putting out there with some skepticism where they were like, is this a real project? Like, is Wes going to lose interest like next week and just move on from this? Because a lot of people are just throwing code over the wall on they're putting up, you know, a 30 prompt project on GitHub, and like, hey, look at this cool thing I made, like, then they move on to the next thing. And, like AI has given us the worst ADD of all time, essentially.
And I, myself, like I've fallen into that trap a little bit, but like, you know, but I think sometimes like, you know, like I started building, I started building Robo rev, like a code review system almost three months ago, and I still work on it almost every day. And I talk to people, and they're like, you're still working on that? Like, I'm like, No, it wasn't. It wasn't just like a vibe could have prototype like I this is serious. Like I use this. This is software that I use every single day. And like, I develop it every single day, because it is like become, it's important to me.
And but like, not all, you know, not all software we're seeing is where new software we're seeing is like that a lot of it's like, you know, yeah, people are having fun with Claude Code. And that's totally fine. I don't judge it. But But yeah, like, I think it's created this problem where like, people are like, people's eyes are glazing over with like, 100s and 1000s of new projects that they're seeing every day. And they just don't know, like, is this a real project? Should I pay attention to this? And a lot of people are, they're just like, their brains have turned off and like, you know, are not engaging, because there's just too many new pieces of software to look at.
Returning to old projects with new tools
Yeah, I mean, interestingly, in some ways, I've had kind of not not in the tools I use, but in the things I've been using code for, for almost the opposite, like, I've been coming back to packages that I've been working on for 10 or 15 years, that have accumulated a bunch of stuff over that time where like, the cost benefit ratio has never been worth like fixing these issues. But now like the cost has dropped so much that I've been having a lot of fun just like I've been working on Roxygen 2 lately, which is a documentation system for R packages. Like I've closed like 150 issues in that and closed them by fixing them, not by like saying won't fix. By and large, yeah, I like that I've mostly, mostly fixed them in a way that like just that that like feels really fun and empowering to me and feels like I'm like, right, like that's making the software better. Because I'm just grinding down all these little edge cases and like toe stubbers that people used to hurt.
I think it's helpful to hear because like, like that project is, I think it's a safe to save like Roxygen 2 doesn't necessarily spark joy in you in the way other projects do, but you feel some responsibility or, or it's undergirding a lot of things. Yeah. Like it's infrastructure. It's not like sexy. It's not fun. Like people by and large don't come up to me and be like, Oh my God, Roxygen 2 changed my life. Um, but yeah, like it's, it is, it's clearly like really important and I get a lot of satisfaction from just like making it incrementally better.
I mean, I think what you're describing is also, there's like almost two, there's probably more like a spectrum than two classes, but I feel like there's infrastructure stuff where it's basically plumbing and I just want it to work. And you know, if an edge case pops up once or twice, I'm like, well, at least I know what's there, right. I can either avoid it or build in a workaround that the need to fix like that cost benefit analysis that felt that. And the other end of the spectrum, I think is sort of more like what you're describing Wes and where I've been with, with my systems is it almost feels like a hot rod where I do actually like I'm drawn to work on it and to see and experience the improvement because I am the primary user.
And there's also something, you know, I feel like, I feel like most people who are, are not just software engineers, but makers in general, there is just the satisfaction of making stuff. Right. And I, I didn't realize how much I was missing that until I kind of got back into it. And if anything, the real challenge has been like figuring out the balance of those things where I'm like, all right, I want to build infrastructure tools that don't just make my life better, but also make like my teammates life better or make us more consistent and those kinds of things. And I don't, if anything, I don't want to tinker with those. I want them to be invisible, the best, that's the best version of invisible software, right? It just does its job.
And then I've got the thing that I actually work on every day. And every time I feel one of those little paper cuts of a rough edge and I go, well, I could file a bug or I could fix it now, how does it fix it now? And there's an interesting discipline to be like, no, no, actually file a bug and then like make a session where you're going to go burn through a bunch of those later that I've had to, I don't know, somewhere between relearn and maybe learn for the first time is something that a more experienced engineer is approaching differently.
What AI coding agents have revealed about software engineering
Yeah, I mean, I think I think what's my my like my thesis on this is that I believe that what AI and coding agents in particular has revealed is that is that coding itself was always something that was a bit of a like a bit of a mechanical act. Once you have a clear picture in your mind of what of what you want to build. And so you write down essentially you have a mental model of what you want to build and then you think, OK, I need to write these tests and I need to set up like these, you know, these structures and these classes and write these functions and things.
And so like that, that like mechanical execution part of of software engineering is now like can be entirely delegated to the agents. But what's left is the thing that was always the hard part, which is deciding what to build and what not to build and having like the clear conceptual model in your mind of like the system and how it should work and like what are its responsibilities and like what's the minimum viable like functionality to be to be useful and then designing it in a way that is like pleasant and like joyful to use.
What's left is the thing that was always the hard part, which is deciding what to build and what not to build and having like the clear conceptual model in your mind of like the system and how it should work and like what are its responsibilities and like what's the minimum viable like functionality to be to be useful and then designing it in a way that is like pleasant and like joyful to use.
And so I think like, unfortunately, you know, the if you just let, you know, Claude Code or Codex rip, it's kind of like going out into the into the desert with a with a speed racer and just like pushing, pushing on the accelerator like you're going to go really fast, but like, you know, where are you going to where are you going to end up exactly? And so kind of without like a, you know, like an operator in the loop to like to steer and make those decisions and decide what to build. Like you could set down two people with completely different backgrounds to work building the exact same thing and, you know, a day later end up with two things that are completely different in terms of like, you know, features and usability and just overall overall quality. It's like same same exact AI models, like very different, very different outputs.
And I think that's something that's being lost in the dialogue a little bit because like, I think I think, you know, if you just read the tech media, they're like, oh, AI is transforming everybody into coding superheroes. But like, as far as I can see, like, that's not the case.
No, and I have a lot of exposure to people who are like on the edges of tech and tech companies but are not software engineers themselves. And I think the gap it's interesting because I think two things can be true at the same time. One is I think somebody with the curiosity to learn and the willingness to push through some of the struggle and the willingness to like build stuff that they throw away as a learning project to build the thing they actually want to learn. I think that that bar has been lowered and there's a lot more people who did not think of themselves as programmers or software engineers because they had no training or experience are capable of experimenting and learning more faster.
But I see way more people who look at this and go, I am I don't even know how to think about like, I don't know how to think the way you all think. I have this kind of working theory. I'm going to test this out. I'm curious how this lands for you.
I think that software engineers and developers more broadly, I think, have a range of skills and traits that we're known for, some more complementary than others. I kind of want to figure out, like, what are the complementary ones that you can learn to think like a programmer, but just the good parts is kind of the way I think about it. So systems thinking, task decomposition, diagnostics and debugging. I think those are skill sets that are widely universal and would make anybody in any field, creative or otherwise, better at their job, probably more successful, probably more satisfied. But we relegate the training on those skill sets to certain domains, including engineering.
And so what I'm not seeing, everyone's like, yeah, just fire up Claude Code and tell it what you want to your point doesn't really it may get you a feeling like you've gone very far, but you're going to hit a wall pretty quickly. And I'm curious, if you were going to, I imagine you guys have been in this career for as long as you have to taught a lot of junior developers. What kind of resources do you point people to when they're like, I'm not just learning how to write code, but I'm learning how to think like a software engineer. And I'm also curious if there's any core skill sets that I didn't mention that really jump out is like, now, if you're going to learn a thing, learn these.
And here's some great places to pick that up, because I guess I'm trying to kind of compose that because I find myself, those people are coming to me and being like, you're not an engineer, but you, you know, this, can you teach me and I want to kind of be a bridge, like I can speak both languages. I can be a bit of a translator, but I can also distill best practices from both directions and hopefully help be helpful in both directions.
Yeah, I certainly don't have any answers. I mean, I've been thinking about this from the side of like the data scientists to like, if we kind of assume, which I think is a fairly safe assumption now, like the ability to write code as much as valuable than it used to be, like you still, you're going to still need, like, if you're constructing a visualization, you still need to understand like how a visualization works, what are the parts? So you can like precisely, so you've got the right language to express that.
And, and more holistically, like, I think a really important part of being a data scientist is like skepticism and like traditionally, like the way you learn that is just by doing a bunch of stuff and like getting it wrong. Like, how can we kind of like shortcut that now or like speed up that process? So you don't have to climb up, you know, all the same steps that we did. How do we kind of give you those, like those, those like mindsets and like attitudes more than like kind of knowledge and skills.
So yeah, I would, I'm super interested. Like, I think there's a lot of like, it seems like a really scary time to be like a junior engineer. I think it is a really scary time to be a junior engineer, but it also seems like we, we can overcome that. We can figure this out. Like what are the new core skills?
I mean, I mean, I think like the, I expect that the computer science curriculums are going to be become much more focused on like studying less about like to use the analogy to like English and English literature and things like that, like more like analysis of things that have been written as opposed to writing itself. Cause like we aren't, we aren't writing anything anymore. Like I haven't written a line of code in six months, like no joke, aside from like some bash commands and things like that. But, but, but basically like rather than learning how to write Java or how to write Python or how to write C++ or whatever, that we're instead going to be studying like software architecture and design patterns and, and things like that.
And from now, you know, using coding agents, you know, all day, every day, essentially for, for many months now, like I am impressed that they do use a lot of technical language to describe what they're doing. And if you're using like, like a spec, like a spec process framework, like superpowers, if you go and actually read the specs that superpowers creates prior to implementing something that uses very technical language to describe like the design of the system that you're building. And so if you don't have the basis to understand, like, what is a model? What is a view? What is a controller? Like, what is the data plane? Like, what's the IO plane? Like, what does async mean?
Like to have like an intrinsic understanding of when, when like the agent is going to ask you questions, like, how do you want the system designed? And maybe it's inclination, its initial inclination is the right one. But a lot of the time, it's not like it will present