On Rails

Jason Meller: Rails, Security, and the AI Advantage

Rails Foundation, Robby Russell Season 2 Episode 3

Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.

0:00 | 1:11:31

Jason Meller, founder of Kolide (acquired by 1Password in 2023) and now VP of Product at 1Password, joins Robby for a conversation about a career at the intersection of Rails, cybersecurity, and building.

They dig into why Rails has become one of the most token-efficient architectures for LLM-assisted development, and why that advantage matters as token costs increasingly shape what's worth building. 

Jason also shares what he's learned about keeping developer environments secure as agentic tools become part of everyday workflows, covering 1Password's open-source SCAM benchmark, how LLMs handle credentials when operating autonomously, and practical steps developers, founders, and engineering leaders can take to stay ahead of it.

Tools & Products

  • 1Password (https://1password.com)
  • Kolide (https://kolide.com)
  • Cursor (https://cursor.com)
  • Claude / Claude Opus by Anthropic (https://anthropic.com/claude)
  • OpenAI Codex (https://openai.com)
  • Lovable (https://lovable.dev)
  • CrowdStrike (https://crowdstrike.com)
  • GitLab (https://gitlab.com)
  • Oh My Zsh (https://ohmyzsh.sh)
  • Wiz (https://wiz.io)

Projects & Benchmarks

  • SCAM Benchmark by 1Password (https://github.com/1Password/scam)
  • OpenClaw (open-source agentic AI tool)
  • Honest Security Manifesto (https://honest.security)
  • 1Password Environments for Developers (https://developer.1password.com/docs/cli/secrets-environment-variables)
  • The Rails Foundation (https://rubyonrails.org/foundation)

Books

  • You Can Stop Stupid: Stopping Losses from Accidental and Malicious Actions by Ira Winkler & Tracy Celaya Brown (https://www.amazon.com/dp/1119566711)

Send us Fan Mail

On Rails is a podcast focused on real-world technical decision-making, exploring how teams are scaling, architecting, and solving complex challenges with Rails. 

On Rails is brought to you by The Rails Foundation, and hosted by Robby Russell of Planet Argon, a consultancy that helps teams modernize their Ruby on Rails applications.

[00:00:03.10] - Robby Welcome to On Rails, the podcast where we dig into the technical decisions behind building and maintaining production Ruby on Rails apps. I'm your host, Robbie Russell, and I run Planet Argon, where for over 21 years we've helped teams maintain and evolve long-lived Rails apps. So I tend to approach these conversations through that lens. In this episode, I'm joined by Jason Mellor, VP of Product at 1Password and a board member of the Rails Foundation. Jason was part of the team behind Kolide, a security product built with Ruby on Rails that was later acquired by 1Password. In our conversation, we dig into the tension that's starting to show up more and more. Rails may be one of the most efficient ways to build software in an AI-driven world, while at the same time, the way we're using these tools is quietly introducing new security risks. We talk about what he's seeing inside 1Password as teams experiment with AI, why Rails might have real features-per-token advantage, and where developers are unknowingly exposing sensitive data, often just through how they work locally. We also get into phishing, not that kind of fishing, phishing, and why these attacks are getting cheaper, more scalable, and why developers are prime targets, and a few practical ways to better protect yourself and your team.

[00:01:11.01] - Robby All right, check for your belongings, all aboard. Jason Mellor, welcome to On Rails.

[00:01:18.02] - Jason Hey, Robbie.

[00:01:18.23] - Robby We got to talk a little bit last year, maybe it was in Amsterdam at Rails World, maybe it was before or after one of your talks there. So I've been looking forward to getting to have a conversation with you. So first off, Jason, what keeps you on Rails?

[00:01:31.08] - Jason Oh man. I mean, I've been working with Rails since about, I think it was right after the merge. So this would be 2009, 2010 is when I first dove into Rails. And for me, it was like this incredibly enabling technology. Like, I had always been a web dev, but I wasn't very good at it. And I found that I would get stuck in the, what do I choose? What's the right way to do this? And I was always looking for like these canonical, this is how you do it. And there just wasn't any opinionated framework out there. And I was introduced to Rails by a work colleague. And I just watched how quickly he built a web app from scratch. I got to watch him do that from the ground up and ultimately fell in love just watching him build in it. And ever since then, 2010, I've only built with Rails. Anything that was web app related, it's only been Rails. And the reason why is I think I've always been somebody who cares about what I'm building in terms of how is it helping other people. Rails has always been a framework that allows me to focus on that. I am a technologist, but I don't want to necessarily relitigate how do I do CSRF protection. I just don't want to do that. I want to know that that's already solved for me and move forward. And then you get into a little bit of the habit of how you think and it kind of combines with the Rails way and you'd be somewhat addicting in that sense because now every time I think of a new idea, I'm still thinking of it as how will this work with a model and a view and a controller. And so that's what's kept me on Rails. And I think there's a lot of exciting reasons to stay on Rails, even now as we progress almost to like 20 years of Ruby on Rails.

[00:03:19.18] - Robby It's been a few minutes now. 21 years now, I think I've been using Rails now. So wild. My Rails career is legally able to drink in the United States. So maybe I need to. So just for some context for our listeners, you know, you're at 1Password and prior to that you were building something called Collide, which to my understanding is very much a Rails application working on solving some security problems. Provide a little bit of context about what you were building with Collide and how that kind of manifested itself into the 1Password umbrella of product offerings.

[00:03:53.03] - Jason Yeah. Yeah. So I've always been a cybersecurity guy. I've been doing cybersecurity. Well, let's just say like even before it was my career, I was one of those 13-year-olds that you dread to meet online, like a script kitty, let's say. But I've always wanted to combine my interest in building web apps with how to be a good person, help companies and individuals defend themselves. Kolide came out of my desire to build something that found this nice balance between the company needs to understand what's going on with all the devices that are signing in to like their most important resources, like their web apps, and the end users, I consider to be their right to have some degree of privacy. And so I wrote this manifesto called Honest Security. That's actually the URL, honest.security, and kind of explaining this tension between this idea of like end users, at least some security folks feel that they are like the root cause of all issues, and my feeling that actually if we are open and honest with end users about what we want them to do for us and we make it really clear to we can actually have them be the thing that solves some of the most nuanced cybersecurity problems in general. So we built a whole product around this called Kolide. And at its core, all you did was go to sign in like you do normally on your work apps. We would check to make sure your device was good. And if it wasn't, we would give you clear instructions on how to fix it. And you could do right there in the sign-in flow. And if you didn't do it, your organization could start warning you and then eventually block you, which then eventually gets you to actually listen and pay attention. I know I'm not the only one who doesn't necessarily always restart Chrome when it's flashing with a big red update. And this is one of those things that kind of can shake you out of it and get you into it. So 1Password saw this and they're like, "Well, we really care about that intersection between humanity and security." 1Password's built on that and felt like we were really directionally aligned. And I had just had twins at that time and I was like, "I can use all the help I can get." I'm a solo founder. So it was pretty good timing. So I've been at 1Password now for 2 years, and thankfully I've been a 1Password user since all the way back, I think in college. And now I have an opportunity to have like influence on this, this incredible product that I love, not only just the things that I built at Kolide.

[00:06:17.23] - Robby I'm trying to remember how far back I went with 1Password. I feel like it's 2000, at least 2008, if not earlier.

[00:06:26.12] - Jason But that's where I was as well, right before OnePassword 3 came out.

[00:06:30.12] - Robby But I remember also back in the day, wasn't OpenID going to solve all our authentication problems at one point?

[00:06:35.22] - Jason Oh my gosh.

[00:06:36.15] - Robby Does anyone even remember what that is? Uh, was that from the LiveJournal folks? I forget, I forget who created that. But anyway, it sounds like you were a solo developer. You used Ruby on Rails to build out that initial Collide product and were able to run that then. So it just kind of seemed like a natural fit to keep working with that. How would you say Collide or Ruby on Rails benefited your developer experience in building a product by yourself like that? Was it up to the hype of the one person framework in that scenario?

[00:07:03.11] - Jason It wasn't by myself. We were a 30 person company by then.

[00:07:06.06] - Robby So the founder.

[00:07:07.00] - Jason Okay.

[00:07:07.05] - Robby Okay.

[00:07:07.12] - Jason Sorry. Yeah. So I was the founder. The thing that it's always been my superpower with Rails just throughout my career is that you can kind of demonstrate something is possible really, really quickly. And then that is really inspiring to everybody who is around you. Trying to help you finish it. And so I think it is a one-person framework, but it's also a really great way to get this energy in the room where suddenly, hey, this is coming together really, really fast and I can see where this is going and I wanna add my own little riff to it. I've never really gotten that same experience as quickly outside of the Rails framework. So as a startup founder, it was incredibly powerful. And the other thing is, it shortcutted all the arguments that every engineer deeply wants to have. Like, are we gonna use UIDs for our IDs for primary keys? Are we gonna use underscores, camelCases? Are we gonna use plural names for this? All that's just decided. So the only thing left to talk about is actually the thing that you're building and why it matters. And so for me as a founder, who had to kind of be across product and engineering, Rails always allowed us to kind of focus the conversation, what we wanna talk about. So from that perspective, yes, I did do a lot of the coding on it, which I wouldn't have been able to if it wasn't in Rails, by the way. And then number 2, we were in good company. I always felt confident that if I built something great, it would scale. There's so many examples of that, and we were right. We now power millions and millions of authentications through this Kolide Ruby on Rails app, and it's incredible to see.

[00:08:40.23] - Robby What parts of the stack do you feel like Kolide has pushed Rails hardest?

[00:08:46.13] - Jason You know, it's so funny, when we were building Kolide, it always felt like right around the time we needed something, it was being merged in by the GitHub team. Because right around the time we had started getting traction in 2019, this is around the same time we had all the great folks at GitHub contributing back all the stuff that they had built in their custom fork of Rails. So the big thing for us was when Eileen coded or merged in the multi-database support. So that was right when we're hitting the fringe of like, okay, we need, we need to start doing sharding or we need at least a reader-writer setup. I think I was able to get that in our app in a single weekend because of how easy that was made. I was able to get that set up and then within the next week we had pretty much like solid horizontal scaling strategy that we still use today. So just something as simple as that, like getting to take the lesson, the hard-won lessons from folks that have run it at scale and then kind of import it into our own project with relative ease. That was great.

[00:09:49.01] - Robby For something like Kolide, when it comes to security, bringing up horizontal scaling, is it kind of like a multi-tenant database architecture? How do you isolate all of this different— maybe in that scenario, I don't know if Kolide stores credentials or—

[00:10:04.09] - Jason No, this is a great topic actually, because I would say like I didn't do— we didn't do anything super groundbreaking in our Rails app, except I really feel that we honed our multi-tenant strategy. In fact, I loved it so much that I posted about it and it went somewhat viral in the Rails community. It was on one of those developer community forums. I think dev.to is one of them. And so it was called the, it's like a 30-line multi-tenant strategy that just works for Ruby on Rails, I think is the title. It was this thing that I happened upon, which is there's this little-known feature in Rails. Well, not little-known, but like the usage of current.

[00:10:42.15] - Robby So current, Was that the uppercase C current?

[00:10:45.17] - Jason Yeah. Uppercase C current. Yeah, 'cause when I've been building Rails apps since I started, like, yeah, you always kind of define like a current user private method in your controller and then start, you know, assign it as a helper and then use it everywhere else. What I needed to do though was I wanted really strong guarantees that when we were in customer B's tenant, the queries would automatically just be scoped to customer B. And so the way that we went about this was pretty clever. We ended up using Current to effectively assign the current tenant after they had signed in, and then we could use a scope within the models to define the boundaries of where they could query. I'm not doing a very good job explaining. The blog post does a much better job. The thing that makes it great is that it is so small. Like you look at some of the other multi-tenant strategies that are out there for Ruby on Rails. They're like full gems and they kind of like rewrite how ActiveRecord works, or they use things like Postgres schemas, which in my opinion, don't scale well. This uses like the tried and true way of just, okay, we're gonna assign an account ID or an organization ID to every single relevant row to the database in every table. And we're gonna have this really smart scoping strategy. And I know a lot of folks, like, they don't like to use scopes and models. This is an example of where you're supposed to use scopes and they work really, really, really well. So as a result of this, this 30-line strategy scaled incredibly well. And it's been the backbone of how our software's been structured. And I remember when we were going through diligence with 1Password, they really wanted to dig into this 'cause this is an area where you get it wrong. And they were pretty pleased by the outcome.

[00:12:33.16] - Robby If you could share a little bit about when they did their due diligence, my company will get called in to do third-party analysis of like an existing Rails app, usually on the buyer side. They're like, hey, we're considering buying this application from this company. We need a kind of independent review of this codebase. Are we buying a piece of crap? Or how big of a mess is this going to be for us to take over over the years? What was that experience like on the acquiree side, I suppose?

[00:12:57.14] - Jason Yeah, so number one, I didn't expect this, but 1Password actually has some really interesting stories around Rails. In fact, the founders, Dave and Roostoms, both specifically said they were inspired to start 1Password because of the release of Ruby on Rails. They saw it, they were just so inspired by what was possible. They went off and they ended up building something that didn't necessarily need Ruby on Rails, but they definitely attribute David as a major source of inspiration in that framework specifically as a reason why they were like, we should do our own thing. Let's get it going. And so that meant like they were coming in with, I think, not negative bias against Ruby on Rails that can kind of exist out there. And the truth is that because Rails kind of picked a lot of really great defaults for us, we didn't necessarily have to go into all these different custom implementations of all these different things like we were pretty much using the Babel things, which made it very, very easy to reason about the code base as a whole. In fact, there were some engineers that weren't familiar with Ruby on Rails, so like, where's all the code? Like, just like, how does this work? And then you have to explain a little bit of the Rails magic to them. But ultimately, I think they walked away with, this is incredible how much functionality you're able to get out of this with so little code. And I think that put everybody at ease and that was a big test, right? You don't know how people are gonna react to the code that you've written. Until you're in that room. And, uh, that was, I think, maybe one of my favorite parts of the acquisition was going through that with the One Password engineers who deeply care and turned over every stone that they could to make sure that we built something that they could stand behind. And, uh, the fact that that went well and it was actually the basis of why they wanted to move forward with the acquisition, that meant the world to me. And it really validated that we made some good choices on our end at Kolide to go with this framework.

[00:14:55.15] - Robby That's wonderful. So it wasn't a case of like, where's all the code for this? There's too much magic happening in this Rails application. And like they understood that and it wasn't necessarily seen as like a downside that Rails makes it easy for things like that. I'm curious about, is Rails used anywhere else? 1Password, is it primarily on the Kolide product at this point?

[00:15:13.13] - Jason Still just the Kolide product, but the reason why we joined the Rails Foundation was we plan on expanding its usage pretty significantly. Maybe I'm speaking out of school a little bit, but I would say that, and I'm clearly biased because 'cause the Kolide team is still like a special part of like who I am. And so I have an affinity towards them, but they've just been crushing it lately. We just released a bunch of new stuff at this big security conference that we do every year called RSA. It's in San Francisco. And a lot of that capability we're able to build so fast because it was the Kolide team bringing it to bear. And as we deeper and deeper integrate Kolide within 1Password itself, there's a little bit more rails in the backend. And as a result, we can move a lot faster. There's areas that we can't use Rails that obviously, 'cause 1Password is an app that lives on your computer, but there's, I think, a lot that we can do on the things that live outside of that, that 1Password still needs to build that are going to give us a boost. So that's, I think, the best way to think about it is we see it as an opportunity to go faster and things that don't necessarily need to be in our existing zero-knowledge framework that we use in our existing 1Password architecture.

[00:16:21.12] - Robby That's great. Maybe we'll come back and talk about this again in a year or two and see if that's changed a bunch and you've got a bunch of other Rails apps there or something. So one of the reasons I wanted to have you on the podcast in particular is you've recently said something that struck me, which was that Rails has features per token. And for our listeners, I'm air quoting that it has an advantage of features per token. What does that actually look like in practice? Yeah.

[00:16:44.11] - Jason So this came up at the Rails Foundation board meeting. And one of the things we started talking about is there's a bunch of CTOs that are on that board, some really great folks, and they're all kind of experiencing this rush of engineers that are now leveraging AI to do their job. What's interesting is everybody's a little bit different in their evolution on this journey, right? There's some folks that I'm sure listening to this right now are still thinking about how do I enable my engineers to use AI quickly. And then there's folks that have already gone through that process and they're dealing with how do I get this to not be a financial calamity? I mean, there's folks that are spending now 7 or maybe even 8 figures in token use, and that's on top of their existing engineering spend. And they're like, okay, mission accomplished. We got everybody to use this thing, but what are we getting out of it? You know, we've now spent like $1 million on tokens. Has that increased our velocity? Are we producing more? Are the customers happier? And I think those are really valid questions to ask. The question that I ask is, you know, we have competitors out there that are also leveraging AI. In a world where everybody's leveraging AI, what is a competitive advantage and what's the right architecture to have that can best take advantage of this AI velocity that you can get? And it instinctually felt correct to me that Rails would be a top contender there. And what I mean by that is, how many tokens does it actually take to produce a meaningful, valuable feature that your users are going to point at and be like, I like this, right? You take the same people, you take a great PM, you take a great engineer, and you take a great front-end person, and you bring all those folks together using the same tools, but under the hood, different architectures, who's gonna go faster? And what we're finding inside of 1Password, and then at this board meeting, we were sort of talking about it, and I think we're commissioning a study as well. We're finding that Rails significantly outperforms other architectures, like that Next.js kind of forward architecture and all sorts of different types of web frameworks that are out there, Rails just token for token tends to produce more with less. And not only just more or less, it's also built in a way where there is an understanding of like the Rails way to do it, and you're not kind of like circling back and trying to like relitigate everything that it wrote again. It feels like it's flowing along with your thoughts. And this is important because if you feel like you're fighting the LLM to get those same outputs and you're frustrated, not only is that going to be token efficient, but you're going to get gassed as a person and you're going to start to not really consider it as a critical part of like your toolchain for building new things.

[00:19:49.16] - Robby I was trying to think maybe back a year, year and a half ago, I remember the conversations around using LLMs to help you with code generation and things like that starting to pop up a lot more frequently, like, oh, this might be a thing that it felt like at the time that because maybe Rails wasn't as, I'm going to say, I'm air quoting, popular or as widely used, there wouldn't necessarily be as much reference data points about source code for the models to learn from and to mimic when they generate code in the future. So like there's way more other, you know, JavaScript frameworks and, you know, I don't know if that's actually true or not, but it just seems to be the general vibe. Like, well, Rails might be at a disadvantage because it's not been the most popular programming language and framework to work with necessarily. Now it's kind of like completely flipped that around being like, well, maybe regardless of that, it takes less code. There's more consistency between Ruby on Rails applications that the LLMs can generate things that look and feel much more like a normal Rails app in theory. Was that something you were hearing back then as well, or thought about?

[00:20:52.18] - Jason What you just said is, I think, what feels instinctually correct before you kind of delve into the details, right? You just like, okay, Rails has had its day, right? It's getting old and these LLMs, they're gonna want to defer to a more modern stack. And thus, 'cause there's so much more, it's going to be better at it because more is better. Turns out more is not better. It turns out there is a lot of really poorly written web applications out there that are being equally considered along with the best web applications that are out there. And so number one, more is not better. Increasing repetition does not necessarily yield better outcomes for LLMs. You know, garbage in, garbage out. So that's number one. Number two is because of the uniformity that exists between Rails apps for the little that is out there, and I wish that there were more. I think that what I've been really appreciative of David recently and Basecamp and 37signals is that they're starting to open source a lot more of their stuff and encourage others to kind of, hey, show us what you got, put it on GitHub. Even if you're not making it an MIT license, let's at least get it out there and allow it to be explored. So ultimately, even though we could use more, the stuff that is out there is pretty good. I mean, if you take a tour of like the GitLab codebase, it's an incredible codebase. I mean, I've learned so much from just perusing that codebase about how to solve similar problems in my own app. And to see like a really high-scale application like that being used as the basis of training, like clearly part of the training set, that's a pretty strong reason why we have good data today. So I think there's some tremendous apps out there. Second thing is that our documentation has always been phenomenal for Ruby on Rails. Going all the way back to its inception, there's a ton of work that a ton of folks that will never be thanked enough to really not just explain how to do something, but why we're doing it. I think this is where that self-taught programmer paradigm really came from. I think Rails is like a part of that. And there's just so much online to explain the right way to build a Rails application. And then the types of folks who build Rails apps, they're always obsessed about, is this the right way to do it? And they're always seeking out, okay, it's been a year, is this still the best way? And they're producing content as much as they're consuming it. So I think there's been a lot for an LLM to ingest there. And then you have things like our CLI, which the Rails CLI always does a great job of shortcutting a lot of the generation of the files themselves, which creates a ton of efficiency. And the CLI itself is really well documented. So ultimately, I think all those things all come together in a really nice way. And of course it was an accident. I don't think anybody was thinking about, oh yeah, it would be great. We do all this in 10 years, LLMs are going to be really good at writing Rails apps. I think that the community has just done a great job of investing in itself and all those investments have created this incredible dividend for us to all enjoy now.

[00:23:57.02] - Robby I've always thought that Rails kind of makes it easier, or it's a cheaper framework to like say think in. And as a someone that's building things, in my case, I'm building these for our clients, but it helps someone accomplish a goal with a thing. And it's always been like, you could focus more on the product. You know, we can argue about or debate like the shades of button colors or other things that would be like, how are we going to like, how the thing actually works? That's the easy part. It's a bunch of CRUD applications and kind of demystified that in a lot of ways and allowed us to focus on all the other interesting things. Do you think there's also just a little bit about the types of developers that the Rails community has attracted over the years that kind of, you spoke about the documentation and people that write blogs and books and stuff like that, but do you think there's like a type of developer that you've noticed that's not like other tech stacks that you've ever had an experience of working with?

[00:24:49.01] - Jason No, that's a great question. I feel very strongly that we've always been aligned with that builder persona. And I think that the thing that draws people to Rails, at least drew me to Rails, is that this is going to enable me to get my idea out into the world. And not only get it out in the world, get it out in the world where I'm not gonna be embarrassed later because of how poorly I built it, right? I'm gonna get this thing out and it'll be able to be just good enough where I can then have folks come to me and help me make it better, but I'll be able to survive those first few months. And that's so crucial to builders, right? They want to be building it correctly, but they don't want to delve into like the whole like politics of why this versus that. They just want the answers. I just wanted the answers. And, you know, David, for whatever you think about him, he's a person who's willing to say he's got some answers. And you may not— not everybody agrees with his answers, but he seemed to me, back in the day when I started picking what I wanted to work in, seemed like he was going to stand behind his choices and he thought about them really deeply. So I have no problem as an individual drafting off of his taste. And I think anybody who feels similar to me likely really appreciates that side of our community and the Rails framework itself. I think there's a lot of folks that are using AI today and also are in that same archetype. And I think Rails is a really great complement to that. I mean, I just talked with a founder who lives near me, he's starting his next business and he's using Lovable right now to vibe code the whole thing and really doesn't know what he's doing on the tech side, but he's built something really incredible. But he's terrified because he knows at some level he hasn't built something that is going to work well outside of the vibe-coded universe he's created for himself. And he's like now meeting with consultants and like, how do I make this real? How do I make this safe? And the answer is like, oh, here's 40 options to choose from for this, this app that you've built. And he was really just looking for that one canonical answer. Like, how do I just do the right thing? And Rails has always been good at doing that.

[00:27:02.01] - Robby You know, I'm thinking about how Rails and AI in particular. So you mentioned that in a recent board meeting that it seems like a lot of CTOs are all talking about how to leverage AI or how much they're spending on tokens and is it paying for itself, so to speak? We're getting a return on all that investment right now. And we're starting to take advantage of AI, like in our terminals, the CLIs and local environments and our codebases. And I'm kind of curious, like, where is 1Password at right now with that? And do you feel like you have a perspective and is 1Password kind of on the same page at this point? Because I feel like I've talked to a lot of leaders at different companies. They're like, they're able to have a little bit of autonomy outside of work to experiment with things, but they can't really figure out how to get that working internally because for a lot of other security purposes or whatever reasons or budgetary concerns or maybe internal politics or resistance from some engineers, it'd be like, not everybody's on board yet or a little nervous about it or like, I'm going to lose my job. How are you thinking about that right now? Maybe you don't have to speak on behalf of all the CTOs, but yourself.

[00:28:04.04] - Jason No, yeah, I'll speak at least on behalf of 1Password. This is like the number one topic happening inside the company right now. We're discussing this because number one, it's clear that we could see the benefits of using this stuff. But on the other hand, we built this mission-critical app that has these deep roots. Like, you need to have like this cryptographic surety that what we're doing is safe and cannot be— it cannot be broken, right? We cannot vibe code our way to a secure version of 1Password. At least there's no working theory where anyone believes that we can. And so how do we make sure that we're using AI in a way that gives our customers more capability, but we stay far away from introducing that style of development to some of the most critical foundations of the product itself. And how do we convey this level of respect that we have for our own product and those things in a way that our customers recognize that we're really thinking about deeply? Because the second we ship one bug, maybe it's not a vibe-coded bug, but every customer will be like, "Ah, you guys are vibe-coding, aren't you?" That's why this happened. And the truth is we actually reason very carefully about what we want to work on with AI and what we don't want to work with AI. Couple all that with how do we just get folks excited, right? There's some engineers out there that are really wrestling with, my whole world is going to change. Like there are folks out there that are afraid to use it or they don't understand how it's going to impact them as engineers in the long term in terms of their careers. And then you have the opposite. You have folks that are using it and they become manic in a way. Like they start becoming like they're not sleeping anymore. And then they show you a bunch of stuff and it doesn't make any sense. And then you realize they haven't slept for 3 days. There's like a whole spectrum of uses. So we're also trying to learn and understand what reasonable balanced usage of AI looks like. How do we make sure that we're still growing as engineers in our capacity to understand the technology that we're building? And then at the same time, how do we keep the core of 1Password safe and make sure that we're not glossing over how important it is and the ways that we've always kept that safe? I don't have a lot of answers, but I will say, if anybody here is listening to what should I do at my company, I think you wanna be encouraging. I think you need to give your engineers a lot of opportunity to experiment and to even do personal projects on the company dime so they can just get used to the tools. And then really try to think about your architecture. What are areas that you can kind of cleave off from like the part that needs to be right? Like whether that's for your company, that's billing or the back office staff tools, whatever pieces those need to be right, make sure that those are kind of firewalled off and then give the engineering and product teams an opportunity to really see how fast they can go in areas where the consequences of getting wrong aren't catastrophic. And so I think a balanced approach happens, but I think you want to be enabling. How do you enable folks to use this stuff as safely as possible?

[00:31:27.15] - Robby Do you think about using these tools as being wildly different than evaluating other types of software development tools? You feel like this deserves its own category? Because I've always been wrestling a little bit with the idea of like, do we create like a playbook just for how we're using AI? But then I'm like, isn't this a subsection of just how we evaluate any tooling that we're going to introduce into our workflow? And like, you feel like some companies might be over internally hyping it as this other thing and not kind of just compartmentalizing that and be like, this is a tool. We're going to learn how to use this and get comfortable with it, but know where we can and can't use it. Like, yeah, we don't use certain tools on just any part of our codebase because of different concerns we have, like in your case, it seems very obvious with security and protecting your customer data and things like that. So how are you seeing that?

[00:32:16.16] - Jason Yeah, I think I would say for me, these feel very different than any other tool that I've used. So the idea that we're going to use the same framework, like, oh, we're going to evaluate this new IDE option and that's somehow comparable to an LLM. Something doesn't feel right about that. I've never used a tool that has had such a profound effect on me as a person, maybe since I used Ruby on Rails for the first time. And what I mean by that is it almost feels like you're taking drugs when you use an LLM. Like the dopamine hit that you get when you put in a prompt and then you get something out that is pretty close to what you had in your head, the first time that happens, I think your brain gets rewired and you create this need to keep going because of the profound impact it has on the engineers themselves. And the fact that we know that it's such a useful technology that we know there isn't an option to say no to it, we have to consider how we're going to live with it. So I guess what I'm trying to say is that LLMs are so big of a deal, they're so impactful that there isn't a world in which we can say no to them fully. And thus we are all going to have to adapt and figure out what does an LLM look like within our organization and inside our software development lifecycle. That's not the same thing as like a tool that you're sort of evaluating and you have the option to say no to. You don't have the option to say no to an LLM today. I mean, there's probably some folks that are listening that are like, I could say no to it, like just watch. But I think that's sort of the exception that proves the rule. I think that anybody who's used these things recently already knows that they're going to be with us forever, and we're just gonna have to figure out what the right way to use them is as a community. And I think that goes beyond just your company, which is why I think it's so important to talk with your peers at every level that you can and understand what they're doing. There is going to be a right way to use this that produces the most good for us as individuals and for the companies we work for. And we need to really dial it in. I don't think anybody knows yet, but what I do know is that we're not going back. We're not gonna roll back the clock to 2023 before these tools started really assisting with coding. So I don't think it's an option to keep your head in the sand.

[00:35:04.04] - Robby But Jason, we're living in a bubble. And then like in 2 years, they've been subsidizing it and they're going to drastically increase the prices and then nobody's going to be able to afford it. And we're all hooked on this drug.

[00:35:14.17] - Jason You know, honestly, I'm not going to— I know you kind of said that with the voice of like, this is— I think there's some merit to the fact that, yeah, people are going to realize that they can't do the types of things that they need to do anymore without tokens. And that's going to have some pretty significant consequences. I read somewhere, I'm not sure if it was on X or an article, but the economic model you want to think about when you think about tokens is this idea of a commodity with unlimited demand. So a good example of this in the world is electricity, right? There is like a set demand for electricity right now. So what happens when you produce more electricity? Well, society figures out a way to use it all. And as a result, you can't produce enough energy in the country or in the world because the energy itself is transformative to the economy. And thus the more energy we create, the more we transform our society to be dependent on it. LLMs, or I would say just generally not the LLM itself, but like artificial intelligence, that we have an insatiable, unlimited demand for. And I do believe it meets the definition of a commodity. I believe it is tokens. While there is some differentiation, they are mostly fungible. And so I think whatever economic models you can apply to energy, you're going to be able to apply this, which means price is going to continue to go up. It will, it will. So the key is how do we as businesses figure out how we can efficiently use this commodity in a way that gives us a competitive advantage to all the other folks that are using it as well? How do we use this more efficiently? And I think your choice of architecture, like the majestic monolith versus microservices, that's gonna be huge. I think using Rails versus another type of web framework that does majestic monolith is gonna matter. I think the underlying language is gonna matter, like Ruby versus PHP versus Python versus Go versus Erlang versus Elixir. All these decisions are going to have a massive impact on token consumption. And we are going to figure out definitive correct answers that give us the most bang for our buck. And I suspect, and I think we will be able to prove it shortly, that Rails is gonna be in that top 5% echelon of got just enough of the choices right to give us what we need. And then eventually there'll be other efficiencies and I can imagine other languages could be even invented that are more token efficient. But I think Rails is kind of all holistically brought together. You have a competitive advantage if you're using Rails and LLMs.

[00:37:55.20] - Robby Tired of paying for too many streaming platforms? One for chat, one for notifications, one for real-time, one for real-time but enterprise? Relax. Switch to Action Cable Family Plan from Rails. One setup and everyone gets their own channel. Mom gets live notifications. Dad gets dashboards, and your teenager gets typing bubbles, and your quiet coworker gets presence. With Shared Channels, the whole family can enjoy the same stream, new comment posted, build finished, and everyone's favorite, someone is online. Stop juggling services. Get one bundle that keeps everyone connected, whether they ask for it or not. Action Cable Family Plan. One provider, one connection, endless togetherness.

[00:38:36.05] - Jason May cause increased background activity, sudden interest in channels, and debates about whether there should have been a page refresh.

[00:38:40.00] - Robby Action Cable, now with the family plan. I'm curious about, you know, I think it's interesting thinking about it on a commodity level, but then also for a long time, I think some of the critics of say Ruby or Ruby on Rails has been the efficiency on the server, just running infrastructure, like, well, we can only process things so fast compared to some other programming languages. So there's a different sort of efficiency, I think, or scalability thing, that narrative that's been around for a long time in the non-Rails communities. But now we're saying like, well, actually maybe the efficiency is going to be on the LLM side to produce the stuff that runs. Do you think that starts to kind of even itself out? Or do you think we're probably going to be spending more as companies on LLM infrastructure or tokens versus hosting the software that we're producing.

[00:39:34.04] - Jason So I just reject the notion that, yeah, you get whatever micro benchmark you want that shows Rails doesn't process as quickly, but I think in real-world scenarios, and if you bring in all the other factors, I just don't think there's any merit to that. And I mean, there's so many examples now of companies that have gotten to that post-unicorn status where they're finding that. I think everybody got spooked from the old like Twitter days and like that, that story just hasn't left our collective consciousness. Maybe it never will. I think what we've proven though, before I get into the AI part of this answer, I think the gross margin on SaaS apps in particular is so high that you're just never really going to run out of money or not be able to buy enough servers to process all the stuff. It's just, it was, it was an argument that just didn't hold any water. Like, if you were running a product that has any level of financial success, you're going to be fine there. And then even if it's a for-fun thing, it's not a business, the amount of money they were really talking about, the differences end up being negligible versus folks that worry about this and they pick what they perceive to be all the right scaling means, and then they build something that number one doesn't need to scale because it's not any good and they didn't have enough velocity to even iterate on the idea enough to even get it to a place where anybody wanted to use it. And so I think that's where I reject that. But on the AI side, I do think that efficiency there will matter. It's not just the efficiency piece, right? I want to be able to see what it wrote and understand it. Right? Like I've used LLMs for a lot of languages that I'm not as familiar with, like Rust. And I gotta tell you, as much as I love Rust, I just was working with it just now, 'cause we use Rust at 1Password. I'm like, I'm barely keeping up. Like, what did this thing actually write? With Rails, and yes, there's some bias, I have some experience, but Ruby is as close to English as you are really gonna get. And it is a language that I can take to a non-technical person and they can kind of get a sense of what's happening there. I think the value of that as you're trying to get some notional understanding of what this LLM is doing under the hood and whether it matches what your intent is, that's going to be huge. And I think Ruby as a language and Rails building like a beautiful DSL on top of that, that further creates that succinct conciseness of what I'm trying to do, I think ultimately will be a major, major advantage. And I think that's where that token efficiency comes from.

[00:42:12.10] - Robby Are you experimenting with a lot of the just the commercial LLM tools? Are you using many of the open source models and leveraging those as well? Do you have a sense of that right now?

[00:42:21.06] - Jason Yeah, so we're trying everything in 1Password. We have a formal enterprise contract with Cursor, which gives you a little bit of an all-you-can-eat buffet. Well, not all-you-can-eat. I mean, from our developer's perspective, it looks like it's all-you-can-eat. But you get to choose a lot of different models. We also have an enterprise license for Claude via Anthropic right now. And we're trying to get everything under the sun. What I'll say, at least in my experience using the different models, particularly the latest batch that have just come out in January and February, I'm definitely getting the sense of certain models help with certain things. For number one, Opus 4.6, clearly immeasurably better at UI than Codex 5.4, by a mile. Codex 5.4 tends to produce UIs that are very boxy and widgety. And Opus just seems to be better trained on like, what does a reasonable user interface look like that gets you pretty close to the ballpark? It's also better at vibing against like an existing user interface that's there and trying to build more things that look like that. So if I'm doing anything frontend heavy, regardless of whether it's Rails or not, I'm using that model specifically. Codex 5.4 is really, really good for large sweeping architectural changes that don't have a lot of front end to them. So if I'm doing a lot of work where I just like, all right, I'm going to write a prompt right now. It's going to produce, I know, like in between 10 to 15 models and I want to make sure I have all the relationships wired up right. Codex, at least in my opinion, does a little bit, has the edge on Opus 4.6, but I continue to find myself going back to Opus 4.6 a lot just because it tends to produce solutions that just feel like I understand them better. And Codex is a little bit too clever for clever's sake at times.

[00:44:13.05] - Robby Do you switch between plan mode or not very often, or—

[00:44:17.20] - Jason I am learning to not use plan mode anymore. Like, every time that I look at plan mode and it suggests it, like, automatically go into plan mode, I read the plan and I'm like, yeah, this would've been fine. And like the amount of times that I'm modifying this plan, it seems to be going down by a lot. In fact, it seems a lot easier if it's just gonna go off and do it. And if I'm not happy with the end result, I can litigate that after the fact versus trying to like polish up this plan. And then by the time the plan is implemented, I'm already at the end of my context window and then it like forgets about the plan midway through. I think getting to a point where we can trust these things a little bit more and then just using the tools or Git itself to just kind of like manage, okay, yeah, that was a bad idea. Let's revert that. I think it's a much better model.

[00:45:11.17] - Robby Does your team collaborate on any like shared skills, like for your agent type skills or anything like that? Do you have like a repository that you're collectively contributing to at this point?

[00:45:21.18] - Jason Yeah, we do. And we use a frontend framework internally at 1Password called Knox. And if we want to use these components, we built an MCP server actually for that one. And it just helps you reason through your problem and then effectively suggest the right components in the right way and writes all the SCSS for you and all that stuff. That's been a life changer for me. I've always been like a classic BEM style. I'm just going to bang it out CSS guy. And I really don't like working within the confines of a specific design system. We use design systems at Kolide, but I've always wanted to be able to play a little bit of jazz on top of that. I feel like I have that feeling back, but the design team now has the ability to convey what's really important to it through the lens of this MCP server, and then we get the best of both worlds. I can play jazz, or at least I believe I'm playing jazz, and then they are getting the outcomes that they want. And like, yeah, we're going to be painting the back of this cabinet and like, we're not going to be misusing a component where we shouldn't be using that component and stuff like that.

[00:46:30.19] - Robby Out of curiosity, are there people that hadn't been regularly coding in your projects that are now contributing code through the use of LLM tooling?

[00:46:42.02] - Jason Definitely. We've seen definitely an increase. And I'm going to raise my hand there. I felt very intimidated by the stack that we had outside of Kolide. And I was learning it. We used Go and a bunch of other languages and I was pretty proficient at them. But to get to the 1Password level of like, you're going to build a feature in 1Password and you better not screw it up. Having an LLM on your side to really be there and help you adhere to the patterns and read all the documentation, keep you on that straight and narrow. It's been enabling for me. And as a leader, it's been great because, and I would encourage anybody who is a leader to consider, if you've kind of divorced yourself from the doing piece, LLMs give you that ability to get back at it and still do all the meetings. I still play with my kids. In fact, I think I'm spending more time with them than I ever have, and yet I'm getting all the stuff done. Feels like a pretty exciting time for leaders who used to be technical and kind of like had to hang it up. If that sounds like you, you should give these tools a try because often the experiences that you've had as a technologist before you became a people manager, they're still sharp enough for you to really steer this LLM to get better outcomes than even maybe some of your junior, senior engineers. And that's, I think, been the superpower. And the thing that I worry about the most too, also at the same time, which is I'm really good at this thing because I still know what I'm doing. I have kids, they'll eventually decide what they want to do in life and, you know, if they want to become engineers, God help them. But what I'll say is, how do they learn the hard lessons that we had to learn that gave us all that world knowledge? Same thing with essay writing, like I became pretty proficient at writing, like, toughen it out and learning all those hard lessons and just like having my hand on my face, just like, how do I debug this thing? And then waking up the next morning to the delight of like suddenly it's possible to solve again. Like, we're going to be potentially robbing our future engineers of those types of experiences, maybe. And I can't tell yet if that's going to be a problem. It feels like it will. And are we just going to burn through all the rest of the folks that are here that already know this stuff? And then we're going to have a major problem at the other side of it. I don't know the answer to that yet. And I worry about it deeply.

[00:49:17.04] - Robby That's interesting. I feel like that's probably a bigger topic than we can fully encompass today. But I think about that too. But I do wonder about the debugging part a little bit of like, you mentioned that scenario earlier around there's that person, you know, locally that's using Lovable and they're like getting ready to launch. Want to launch, but they got to get the last X percentage and they want to feel confident about it. Sometimes people just need to launch the thing and deal with the realities of what happens. That's how I learned was just, I didn't have a choice but to figure out the things as they popped up. But now maybe it's a little easier that I can just copy and paste that error stack trace into an LLM and be like, what is this? And it's like, it might be this. Like, that's great, but—

[00:50:00.23] - Jason I definitely don't miss the toil of like, I can't figure this out and there's no results on Stack Overflow and I'm just gonna have to suffer now for like 4 days straight and remember how to open up like a debugger and like trace through like, what does the C binding do? Like, I think we've all had to live some version of that and it stinks. And it's like, that's not, we don't want that anymore. Like, definitely make that go away. The one thing that you just brought up though, that I did wanna touch upon quickly, is another thing that we're learning at 1Password, and I've learned specifically, is constraints are good. And what I mean by this is when you don't know how to do everything, especially if you're a builder of one, you have to constrain your idea. And those constraints often yield really creative solutions of like getting to the pure version of whatever it is that you're building. One thing that I've seen, and you reminded me when you brought up that entrepreneur that I mentioned, is he doesn't have those constraints because he could build anything. And thus the idea is very overloaded in ways that it doesn't feel like an MVP. And I think like that's a trap that we all as entrepreneurs and founders or teams of one need to prevent ourselves from falling into, which is knowing when to stop. And get it out there. And yeah, you get this dopamine hit of like, ah, just one more feature. Oh, what about this edge case over here? The amount of entrepreneurs, if you actually talk with them at the beginning of their journey, they always released what they had often in like anger or with fear and doubt. And then they ended up learning something. They ended up learning they probably built something that was only 40% the right thing to build. And they had that one kernel of like, this is the right thing. And then they go off. And then that's the whole premise. If you end up building every possible variant of every feature that could go into the version 1 of your idea, you might actually rob yourself of the opportunity of a human being feeling like they could give you that feedback that's gonna lend you in the right direction. You're gonna see that you're so over-rotated the wrong way that they're just going to give you that kind of, "Oh yeah, it looks cool. I'll check it out later," response. So that would be like my one warning for entrepreneurs that are using LLMs. Know what is like the pure simplest form of the idea and get it out there before you build every possible feature that could exist.

[00:52:27.01] - Robby You know, that reminds me of, you know, for a long time we would work with budding startups and be like, all right, we want to build this new product. We want to build the MVP. I'm talking, this is 2005, 2008 era where hiring like a full-time CTO, like a technical co-founder was like a nice to have for some companies. It was like, we're just going to outsource it to like a software consultancy, like my team. And so we built a lot of those projects and the constraint always ended up being what can they afford? And that was like the best it was going to be. And so it was always this kind of like, oh, can we get a couple more things? Can we get a couple more things in? I know we're kind of running low on our budget. Can you just give a little bit of equity or something so we can get these other things? Because they were afraid to release the thing. And then I always wondered how that shifted once people started bringing on like a technical co-founder, where then as long as you're willing to keep going before you throw it out there in the market. And now you've got LLMs, you can just keep a couple more prompts, couple more prompts. I'm going to keep spinning up some more ideas. Like, when do you finally be like, this is enough to share and get feedback on and see if I can actually sell or attract users to this thing that I've been working on? It's an interesting thing. Coming up with a budget maybe is a good idea or constraint, as you said, constraints there. One of the other reasons I really wanted to have you join us on the podcast was to talk about security as developers. And maybe about a year ago, you and I had had a side conversation about how like phishing attempts and how developers are often a prime target of phishing attacks. And we were kind of earlier on in the AI thing. And you had mentioned that like, well, because of AI and LLM tools, it's really cheap to get things like close to real time. Do you remember what kind of like touching on there? But in terms of like, why are software developers a potentially good target for hackers, script kiddies to try to come after? I wanted to kind of like touch on that a little bit. Like, can you tell us a bit more about that?

[00:54:17.04] - Jason Yeah, absolutely. I still believe this even now a year later. And I think the problem has actually gotten pretty significantly worse. Developers often they're very good at taking some of the most sensitive credentials or secrets or pieces of data and like all combining it on their local computer, right? Because as a developer, you work at a big company or a small company, you need to fix bugs and you need to fix production bugs. And the best way to do that is have access to production, be able to actually test the bug with real data. And just that scenario tends to happen, and all this data suddenly lands in one place. And the mental model that we've always sort of had, and security teams still operate like this, is that as long as that's on a managed device, we're going to feel pretty good about that. We have antivirus running, you know, endpoint detection response product like CrowdStrike or whatever, and maybe we're using 1Password. Like, we're feeling good about this managed device, and thus we're okay if you have some secrets on here, if you have an SSH key that gets you into prod, or if you have like a bunch of API keys in your shell history or whatever, right? You have a script with one hardcoded in, like we're not going to flip out about it. I think the calculus of this has changed pretty significantly. And I would say in the last 3 months, even more so. These LLMs, they are like having another person use your computer along with you. These tools don't just live in your code editor. They pretty much have access to every file in your file system. Here's an example. There was someone I was working with and they were trying to get the LLM to sign into a web app. And it was able to sign in because it had the username and password was already preloaded into the website. It was just driving the web browser. But then it got the MFA prompt, like the two-factor prompt. And I was like, okay, like the LLM is gonna be defeated by this. The LLM was smart enough to think, you know what? I bet you this user stored a two-factor backup code in their downloads folder. And I'm just gonna check to see if it's there. And it found one and it put it in the MFA prompt and it was able to sign in.

[00:56:38.21] - Robby Oh my gosh.

[00:56:39.22] - Jason And it did this unprompted, right? So these things have full access to your file system. And even when they're operating without malicious intent, they can just grab these things and they can just throw them anywhere that they want. This happened with OpenClaw, right? So OpenClaw got released a few months ago and then someone decided, you know, it would be a good idea, let's build like a social network for just all these OpenClaw bots to just talk to each other. Well, that went viral. They started doing it and what did they share with each other? Plain text secrets that they were just sharing from like their host computer and they were just sharing it out in the wild. And there's like a bunch of talk like, oh, this is all fake stuff. It's just like a bunch of trolls trolling trolls. But Wiz did a great story on this, wiz.io, where they actually dumped all the credentials and they found a significant portion of them were real and that those companies were compromised because OpenClaw just indiscriminately shared them. 1Password, we built an open source benchmark called SCAM, which is like a clever acronym for Security Comprehension Awareness Measure. But the idea is like, if you give an LLM access to a password vault in like kind of an unsafe way, just let it get passwords from the password vault, will it use those? Will it wield those passwords in a safe way out in the wild on the internet? And the answer is no. What's fascinating about this is if you ask an LLM directly, I'm going to show you two emails. 'Which one is phishing?' It will get the phishing email almost like 99.99%. It'll get it right every time. But if you ask an LLM, 'Hey, could you like check my email and respond to some of the urgent things that are in there?' It'll like almost always fall for like pretty obvious phishing scams. So this idea of like this bias that we have as humans, like when I'm prompting you in advance with, 'I'm gonna test your ability to like understand phishing,' you're going to do better at it versus if you're just going throughout your day and you encounter a phishing email or some kind of scam online. So we did this and what was so funny, even the larger models, which you think would be better, they're explicitly trained against cybersecurity. They did a little bit better. Opus had this one moment where it realized it gave a password to like a phishing website, and then it submitted it, and before it returned control back to the user to like say, "Okay, I'm done," it realized after it hit submit that it screwed up. It was like, "Oh no, I just submitted your credentials to a phishing website. You should probably reset those." And so you can check that out. If you just look up, I think it's on GitHub, github.com/1Password/scam, you can actually go and watch the live replays of this. So that's, to make a long story even longer, developer devices are filled with this stuff. And thus developers themselves are really, really good victims because they often yield really, really great outcomes for the attacker who's really trying to actually pivot into the infrastructure and make it a much larger compromise. We just saw this recently with the recent JavaScript issue where the creator of the package that got compromised, it sounded like almost like a spy novel. Like he got pulled into this fake world where he thought he was talking with real people on Microsoft Teams, and it was like that the client was asking to install something to make Microsoft Teams work better. He just like was like, yeah, this seems all legitimate, and then he installed it, and then boom, they got all the signing keys for all of his npm packages, and then they polluted them with malware. That created massive amounts of compromise to anybody who uses this package, and that was like a pretty core package to the ecosystem. So yeah, I can't expound enough about how big of a deal it is. Like, developers have to protect themselves.

[01:00:39.05] - Robby You know, it's interesting because it's not necessarily Rails related, but because I'm one of the maintainers of a project called ohmyzsh, which is used on millions of developers' machines, Microsoft and GitHub have been for the last few years been really I think good at reaching out and like talking with the maintainers with us about security hygiene and trying to make us aware of things. But it's something I feel like that I haven't heard come up in our conversations in some of the trainings that they've paid us to do as well, because they're like, if my Robbie Russell account gets hacked, then, you know, that could be problematic. Not that there's a lot of other platforms that could happen with as well, but it got me thinking like as an employer, I already have my team members occasionally be like, hey, did you send me this Telegram message? And I'm like, no. And then that stuff happens all the time already. Then I'm like, when does this start to impact my personal life? Like, what's it look like to have like my wife get contacted and like her thinking that she's interacting with me? And then maybe there's some way that she gets access. And I'm just thinking like there's other ways that people could— I'm not saying that I'm a target, but I'm thinking like as developers that have access to a lot of sensitive data on my computer, but also access to other infrastructure things that could impact a lot of people. Like, do you have any advice on how—

[01:01:57.04] - Jason Yeah, well, first of all, I want to disabuse you of the notion that you're not important enough to be a target. I think that's the scary part of these AI tools, is that hacking is very much like a criminal enterprise, and it's really a business. And thus, you need to— it needs to be profitable. And if the amount of effort it takes to deploy an attack exceeds the amount of money you're going to get from it, then you're not gonna do that attack. What AI does is it's this great equivocator in the sense that if I have a viable attack that would've normally been very sophisticated and time-consuming to run, if I can automate that really sophisticated multi-stage attack with AI and I can run it on a much larger set of victims, I'm gonna do that because I'm gonna make more money from it. I think without AI, a lot of the attacks that you just described, like phishing is a good example of it, requires someone to really pull together a bunch of like voice samples of you and all this stuff and like synthesize it and like they're going to spend hours and hours on that, you know, maybe develop bespoke malware to actually make sure they're getting the data from your device once they, whatever, whatever the second stage of the attack is. AI likely dramatically decimates the amount of money that they need to spend and time to coordinate that type of attack. And it does make it viable to do that type of attack on a regular everyday person. Or if it doesn't today, it will be soon. Like, we're talking in a matter of a year or less. So my advice is, if you haven't had this conversation with your friends or family, come up with your code word, right? So have a verbal code word. You can always ask me in an emergency situation, like, what's the password? And come up with like a nonsensical word that you're going to use with your spouse or loved ones where you can all do that. Now, of course, that requires you to remember it. And all these scams, they're always around urgency, like, I'm in jail, I need you to come bail me out, I have a flat tire, I don't know where to pull over. We also have to train ourselves to recognize that if we're feeling rushed or there's urgency around it, this is when we're the most able to be victimized by these types of attacks. We're not paying attention, our adrenaline's rushing, our blood is rushing through our ears, we don't notice all the problems with like, the voice isn't quite right. It sounds like the podcast version of you versus the real version of you. So we have to also like train ourselves and our loved ones to recognize the other aspects of the attack, which are around creating this artificial urgency and to make it feel like it's an emergency that they need to deal with right away, or there'll be life or death consequences if they don't act right away. Those are all the things to look out for, and having a verbal code word there. I think there's some technology solutions that are being batted around. But there isn't one that I would recommend yet because I think that simple voice phrase is good enough to defeat most of these. And just having that conversation with your family is a great opportunity to just connect with them.

[01:04:52.17] - Robby Should each of you have your own code word? Are you trying to match the code words? Just want to get into the practicalities of that.

[01:04:57.21] - Jason Yeah. So you say, you know, applesauce or something, right? And say like, I honestly think you just use one if it's for your spouse. That's what we do. And by the way, mine is not applesauce. Okay. Any scary people are listening, but it would be something like that, right? Like what's the code word for an emergency? And I think if you have kids, this is also something that you should teach them. And if you run a business though, and you have a CFO or you have someone who does the books, that's another one that you should focus on. They're often the victims of these types of attacks as well. On the developer side, the thing that I would recommend, and this will be a little bit of a plug for 1Password, but I'll say it anyway, 'cause I think a lot of people are 1Password users anyway, and you can just use this feature. We have a new feature for developers called environments. And basically all you need to do is point 1Password at an existing environment file, like a .env file, where a lot of these secrets tend to live. And we will import them automatically into 1Password and then we will mount on your file system a FIFO Unix pipe version of that file. And so if anything tries to access that file, let's say an LLM, it'll actually make 1Password pop up and you'll have to use a biometric for 1Password to then pipe the actual contents over. I can't tell you how many times this has saved me personally from an LLM just inadvertently trying to rummage through my file system and get at some secrets. We also have a CLI that allows you to use placeholder values and we substitute them in with the real secrets through a runner. That one's more situational. The environments one is just a no-brainer and you get it for free if you use 1Password already and you pay for it. So if you're a developer and you want to take advantage of that, you should use it. And feel free to, folks who are listening, reach out. There's things that you're using 1Password today to solve for because you use our SDK or APIs that you think we should build into the product, like, I'm the guy to talk to. We can absolutely build it right in.

[01:07:02.09] - Robby Well, definitely I'm going to look that up and maybe share a link to that with our listeners as well. Out of curiosity, is there, outside of the air quoting technical world, is there a book you recommend to people?

[01:07:15.04] - Jason Oh man, uh, yeah, there's a bunch. This is a really in the weeds book, but the book is called You Can Stop Stupid: Stopping Losses from Accidental and Malicious Actions. So I love this book. The author actually reached out to me after I wrote the Honest Security Manifesto, which I mentioned earlier, and he was like, you need to read this because what you're trying to say in your manifesto is something that we've been deeply talking about in like the safety science and behavioral science world. There's a story in it that I love, I'll convey really quickly. Back in when the Industrial Revolution started, there was this theory around workers where there was this belief that there's certain types of individuals that are just accident-prone. And you need to identify these people and you need to get them out of your organization 'cause they're gonna get hurt, they're gonna create liabilities for you as a business. And so what they used to do is used to give you a punch card and every time you made a mistake, they would punch your card. And when you filled out the card, you were fired. And this was like the best we could do back at the turn of the 20th century when we were thinking about how do we deal with safety in the workplace. Like these are the best ideas that we had. And of course, over the course of the rest of the 20th century, we actually figured out that there's a whole science behind safety. There's safety science, behavioral science, and deeply understanding how humans can actually, using tools and tactics and techniques, get them to be safer and understand that when humans create an error, it's not the human's fault. It's like a deeper look into a flawed system. This to me is huge from a security practitioner perspective because we still live in a world on the cybersecurity side where we still blame victims. Like that JavaScript thing, there's a ton of people out there just dunking on that guy because he made a mistake instead of really thinking about it from a systems perspective. What could we have done from an npm.js package perspective to make this safer? How do we prevent this from happening in the future? Every other industry knows how to deal with this, and for some reason, cybersecurity, IT industry, we just haven't evolved yet to figure this out. I came from a manufacturing background, and I think the first thing that I learned when I was on the floor of the machine shop was lockout/tagout. If you're gonna repair a machine, use this apparatus to physically arrest the ability for the machine to be turned on while you're back there. And only me with a key and somebody else with a key can unlock it. That is a physical device that has actually prevented countless deaths because someone invented it and stopped blaming it on people just being lazy and not checking if somebody was repairing the machine or not. OSHA as a government agency was created because of that specific problem. There are solutions. So I would recommend people read that book.

[01:10:17.12] - Robby You said that was You Can Stop Stupid.

[01:10:19.20] - Jason You Can Stop Stupid: Stopping Losses from Accidental and Malicious Actions.

[01:10:23.19] - Robby That's a mouthful. Okay, well, definitely look that up and get links in the show notes for everybody so they can check that as well. And I'll probably grab a copy of that myself. Where can folks best follow your thoughts and ruminations about software engineering online? Is there a 1Password engineering blog? Do you have your own blog as well?

[01:10:41.09] - Jason I don't write that much, but when I do, I want it to be on the 1Password blog. So just blog.1password.com. You'll see me writing a bunch. You probably saw a bunch of articles around OpenClaw recently. I was pretty opinionated about that. And more articles about AI and how we're using it at 1Password are forthcoming. So take a look at there.

[01:11:00.06] - Robby Excellent. Definitely include links for everybody. And thank you so much, Jason, for stopping by to talk shop with us on On Rails.

[01:11:05.22] - Jason Absolutely. Thanks for having me, Robby.

[01:11:10.18] - Robby That's it for this episode of On Rails. This podcast is produced by the Rails Foundation with support from its core and contributing members. If you enjoyed the ride, leave a quick review on Apple Podcasts, Spotify, or YouTube. It helps more folks find the show. Again, I'm Robbie Russell. Thanks for riding along. See you next time.

Podcasts we love

Check out these other fine podcasts recommended by us, not an algorithm.

Maintainable Artwork

Maintainable

Robby Russell
Remote Ruby Artwork

Remote Ruby

Chris Oliver, Andrew Mason, David Hill
IndieRails Artwork

IndieRails

Jess Brown & Jeremy Smith
REWORK Artwork

REWORK

37signals