SAM: Good morning. As we record, this is January 25, 2017 and I would like to personally welcome you to our Cyberpunk Dystopia. I’m Sam Livingston-Gray. JESSICA: Sam, that’s a little too close to the truth. Can we just be Greater Than Code? SAM: Greater Than Code it is. Welcome to Greater Than Code, everybody. JESSICA: Thank you and I am thrilled to be here today with Jay Bobo. SAM: Booo! JAY: I’m so excited to be back. I know I’ve been gone for a while but I’m here. It’s going to be a great podcast because all of our podcasts are amazing. We have André Arko and Carina C Zona. ANDRÉ: Great to be here, Jay. CARINA: Yeah, I’m really excited about getting to be on Greater Than code. Thanks for having us on. JAY: Carina C Zona is a developer advocate and certified sex educator. She spent a lot of time thinking about the unexpected cultural effects of our decisions as programmers. Carina is also the founder of CallbackWomen, which is on a mission to radically increase gender diversity at the podium of professional programmer’s conferences. SAM: André Arko has been writing Ruby since 2004, created the jQuery Rails Gem and joined the Bundler core team before 1.0. He founded and runs Ruby Together, the Ruby developer trade association and today he leads the combined Bundler and Ruby Gems team. Welcome to the show, Carina and André. ANDRÉ: Thanks. It’s great to be here. CARINA: Thanks. ANDRÉ: I’m actually super excited to be on Greater Than Code because this means that Amy can no longer hold her show over me. Now, we each have a show. SAM: Well, unless hers is better. ANDRÉ: I’m sure, it is. [Laughter] JESSICA: Oh, Amy Wibowo? ANDRÉ: She’s my partner. JESSICA: Oh, cool. Origin stories! Origin stories! SAM: Yes, please. Who are you people? JESSICA: How did you gain your superpowers, meaning learning how to code and everything else that you are proud of? SAM: And you may say to our listeners as well, how did I get here? [Laughter] ANDRÉ: I regularly asked myself, “How do I get here?” I guess as you just mentioned in my bio, I started doing Ruby in 2004, when it was completely useless. With no reason to Ruby at all, Rails didn’t exist. Libraries were scarce on the ground. If you wanted to reuse someone else’s Ruby code, you had to find their home page and download a tarball and stuff it into site Ruby yourself. That was fun times. But I was in college and doing a CS degree undergrad and I just really wanted to write some code that was more enjoyable to write than C++, which is what all my classes were in — SAM: Hear, hear. ANDRÉ: — And so I guess at the time it felt like obscure and/or unusual languages. Ruby just struck me as incredibly enjoyable to write and incredibly enjoyable to read and I love how much it felt like a fun activity so I started doing a ton of Ruby. I read through the entire Pickaxe twice before I even had the Ruby Interpreter installed on my machine. It was fun times. Then after I’d been playing really enthusiastically with Ruby for a while, I think my very exciting and meaningful project was I tried to copy the interface of iTunes in a web browser as a CGI script. Then on the backend, it executed Apple scripts to control iTunes on my [inaudible] server. It was very exciting. JESSICA: So you ported iTunes to the browser? ANDRÉ: Oh, no. It’s just the interface window, like it listed songs, then it let you filter and it let you play, pause, skip, back, change volume. It never worked that great but it looked a lot like iTunes and as long as you were okay with the delay of running everything via OC script on the command line, which is four seconds and it worked. Then I saw this email on the Ruby talk mailing list from this weird Danish guy named David-something and he was like, “I just released –” I think it was Rail 0.10 and I was like, “Oh, that sounds cool. I’ve been struggling with Ruby CGI scripts. I would like something that was easier than that,” and then I started using Rails and swore a lot to myself as I started using it and it was amazing. I guess, history and inevitability let us here. SAM: But they were good swears, right? ANDRÉ: I said a lot of good oh-wow-this-is-amazing swears as I tried out Rails. JESSICA: Oh, so it was like, “This was a holy shit.” Not, “What the fuck do I have supposed to do here?” ANDRÉ: Yeah, I said holy shit to myself a lot as I was trying out Rails, in comparison to that CGI script that I had been using. Then, it’s history and inevitability. Now, I still write Ruby because I really enjoy writing Ruby and I still pay my bills by writing Rails apps really. JESSICA: Carina, origin story. JAY: Right on. CARINA: How I got started is that I have a very eclectic background, meandering through various college majors and careers. I started off working my way through college by doing office work and found that there was a lot of need for various database stuff. I was doing a lot database development, got into relational databases. It was totally all about, “Oh, my God. You can do that. You can link things together like this,” and start doing scripting for those until finally I reached a point where I needed true programming in order to accomplish things. I started learning purely to solve those basic business office problems that were going on and little by little, I started inching my way into this being something that I could do for a living. The funny thing is that in college, I was actually studying theater but I was at a school that had a very good theater department and a really good CS department. I found that at night, I was hanging out with my great CS buddies instead of my great theater buddies. That’s a little cue about where my real interest probably are. Although, now as a developer evangelist, I think it really circles back to using both of those that I’m able to use my theater background and my programming background and my passion for both to really be able to talk to people about things that I think really matter and that aren’t getting as much attention as I really believe that they should. At this point, I have I think some reputation as a speaker and that wouldn’t be possible without having that convergence of both backgrounds. I love where I come from. I love those roots. In the meantime, also because I just love to do too many things at once, I was also in my spare time doing work as a sex educator. That slowly became things where I was getting certifications and that’s also really merged. If you look closely at my talks, almost all of them have a thread of sex education influences going through so it’s really my way of sneaking in that knowledge to people as well that it’s about things like how we think about gender as programmers, how we think about people’s relationships, how we think about a lot of things about the human side and make assumptions that aren’t nearly as valid as we think they are until we have a good grounding in sex education and realize the world is a lot more complex than we initially imagine, so taking some of the limitations of our education because so many of us are very grounded in CS and not nearly grounded enough at broader range of subjects and really trying to introduce more of the social sciences side of things. JESSICA: Yeah, that’s awesome. In sex education, you must think about culture a lot. There’s a lot of dealing with what is a scientific fact what is a cultural… I don’t want to say decision. What is a cultural — SAM: Norm or default, perhaps? JESSICA: Yes, exactly. CARINA: I would say both of those words. There’s norms and there’s decisions that we make about what we think as a norm or what we think should be a norm. Certainly, that’s sometimes the case. As programmers, we’re deciding, “Well, that’s too much of an edge case so regardless of whether this exists, I’ve decided the norm for this code base is –” so you’re going beyond acceptance of what is typical and saying this has to be typical. SAM: Right and there are ways that we can design things that deliberately include or exclude certain cases like how we design our database schemas around marriage modeling, for example. It’s one topic I’ve seen you use a lot. CARINA: Yeah, there’s that really classic term of the title but it’s — SAM: Schemas for the real world, I think it was? CARINA: No, the article. But it’s called ‘Gay marriage: the database engineering perspective’ and it is at a link that we will provide on the website for this episode. ANDRÉ: It’s definitely a classic of the genre at this point. CARINA: It goes through every possibility. It winds up with Graph Theory, which is the first time I learned about Graph Theory. But he just breaks it down so well or she actually, I’m not really sure what was wrote it. But it’s one of those things that made my jaw drop for the first time and realized, A, this is legitimate topic in programming and B, I could actually talk to other people so they’d care too. JESSICA: Have you mentioned that you got excited about relational databases because you can link these things together and certainly, graph databases are like that too, in another level. I just learned Neo4j like a week ago so I’m having fun with that. I stopped halfway through the tutorial and changed my little program so it will drive dependency graph of the various repositories that I have downloaded. ANDRÉ: That’s amazing. SAM: Well, while we’re on this topic of super important, yet also interesting and entertaining links, one thing I ran across recently was that somebody has gone and collected a bunch of falsehoods programmers believe about X articles and collected them into a meta list called ‘Awesome Falsehoods’ and I’ll put that link in the show notes as well. It’s really cool. ANDRÉ: I am a big fan of that. I guess this predates people putting lists of things on GitHub but I created a meta list of falsehoods programmers believe and put it on my blog in, I think in 2012. I definitely am a fan of this genre of things to think about. CARINA: I wonder if has anything to do with the fact that you have an accent in your name. ANDRÉ: That definitely helps me run into some of the falsehoods that programmers believe on a regular basis. I went to a Meetup at Stripe last night and my pre-printed name tag said A-N-D-R, Unicode thing that stands up and down and then have an intersecting halfway line and then the meta character that represents return at the end of a line, it was pretty amazing. I’ve never seen that one before. SAM: It’s fantastic. I get some decent ones as a hyphenated but yeah, the accent must be even more exciting. ANDRÉ: I am intimately familiar with what happens when you render the Unicode accent with ‘e’ entity in the Windows Latin 1 Encoding, which is capital A with the tilde sign over it and then a copyright symbol. JESSICA: One comment on your perspective origin stories, I thought it was interesting that Carina got into programming driven by the problems she wanted to solve and André was in it just for the fun, at least in Ruby. “I read the book twice before I will write a program.” ANDRÉ: Those was amazing. JESSICA: Carina, what exactly is Ruby Together? CARINA: Ruby Together is a nonprofit trade organization. It’s incorporated in the US. It’s a 501(C)(6), which is not the same as a more commonly known 501(C)(3), which is a charitable organization. This is a trade association. The idea here is that everyone in the industry is pursuing a common goal, shared needs, shared interests, and shared investment solving those problems. Ruby Together’s mission is to support the development of core Ruby infrastructure projects that are open source and to make sure it’s financially viable to keep those going, to be not just maintained and secure but also moving forward, adding new features including things like gemstash and ultimately, in the long-term to be able to pay developers to work full time, not just the infrastructure projects but even eventually things like having a full time community paid developer on Ruby core, on Rails core. That’s a future that we envision. It’s going to take a while but we’ve definitely made progress in just the year and a half that Ruby Together has existed. That’s the short term where we are and that the long term where we want to be is that some things are just too important for the whole community to be controlled by one or a few companies. We need to make sure that we can always be putting community needs first when you’re talking about infrastructure. That’s why it needs to be a community-based organization but we feel really strongly that it’d be something that’s widespread community funding, not dependent on one or a few companies. If you think about potential, if you’re only funded by one or a few companies, if the industry experiences a downturn or one company has problems and starts having to slash its budget, we cannot allow Ruby infrastructure to just suddenly be having problems. There has to be a wide enough funding base that no individual company’s problems become a disaster for the whole community. That’s the kind of where we’re coming from is there is a part that’s ethical and there’s a part that’s just plain all business sense. SAM: It’s worth pointing out that this is not a theoretical thing. This has already happened. There was a time a couple years ago when several members of the Ruby core team and one person who was also a member of the Rails core team were all working at AT&T Interactive and then there was a re-org. Suddenly, all of those people had to find new jobs. Fortunately, I think at the time it was Heroku who has stepped in and started hiring some people. A couple of those people went to other companies as well. But we’re still kind of in that same situation where as far as I know, we’ve got a couple of Ruby core people working at Heroku and other people may be scattered elsewhere. But we’re still highly dependent as a community on corporations who are willing to pay somebody to work full time on language infrastructure. JESSICA: So for Ruby Together, a company can contribute to Ruby infrastructure without the cost of a full time, highly paid developer? ANDRÉ: Exactly. CARINA: Yeah. SAM: Speaking of contributions, you mentioned that it’s a 501(C)(6) not a (C)(3), does it still have that same tax exempt status? Oh, sorry. Does it still have the same property where if I, as an individual donate the contribution is tax deductible? ANDRÉ: That’s a really good question, Sam. To answer the first half of that question, yes we are a non-profit, at least in the US. The IRS said that we meet their criteria for a non-profit organization and to answer with second half of the question, yes it’s tax deductible — asterisk, footnote — It’s not a donation because we’re not a charity and you need to be a person or company that uses Ruby to become a member of our trade association. If you give us a donation — JESSICA: A contribution maybe? ANDRÉ: Right, internally they’re classified as a membership fees. You become a member by paying us a fee and we use those membership fees to fund work that benefits everyone in our sector of Ruby developers. As a result, the way a trade organizations are structured, at least under US tax law is that it’s a business expense for a Ruby developer or a Ruby company. Shared infrastructure is a business expense, in the same way that you buying a software tool is a business expense. If you’re not in the US, I guess I am not a lawyer or an accountant so I can’t offer you a guarantee, but I have heard back from developers in Europe, Australia and New Zealand that it can, at least be considered a business expense, according to some companies’ lawyers and accountants. But if you’re outside the US, I would say honestly, you need to check with your own lawyer or accountant. JESSICA: I think you just said that if I get a membership in Ruby Together, it is not a tax deductible donation but it could be considered a business expense for me? ANDRÉ: That is correct. It is a tax deductible business expense for you to be a member of Ruby Together and then we don’t have to give up a big chunk of that money that you just gave us in taxes. We’re able to spend the whole amount of that money on Ruby developers and infrastructure. JESSICA: If I were to become a member of Ruby Together, what and how often would I pay for that? CARINA: The answer to that is that fees are charged on a monthly basis. Although, you are welcome to pay on any longer basis if you want. A couple of companies just pay once a year because frankly, it’s easier for their accounting department to write one check, rather than deal with it every month. We’re happy to take money in larger chunks of time, if that makes more sense for individual company. But it’s charged on month to month basis. That was really a very deliberate choice so that unlike, say something like a Kickstarter, where you’re funding based on faith in the future, that thing that I’m funding, I hope will pan out, whereas with Ruby Together, if you don’t see at the end of the month that this is going the way you think it should, you can cancel your membership. But we believe that with our transparency every month reporting on what we’re doing, how much money we’re taking in, how much we’re spending, what has been able to fund, what progress we’re making toward the next goal that you will want to next month, continue to pay your membership fees. The structure is really about faith in the community. You put faith in us and we put faith in you. I think that’s very different than most of the structures you see for there’s so many different ways to fund open source. There are dozens and these choices have been made really deliberately around certain principles. That’s a core one. There’s various different levels that you can contribute. For individuals, it’s really low. It starts as little as, I think $10. Business membership start as little as $50 a month and we encourage companies to consider the upper-tiers like $2000 per month, there’s $5000 per month. For large companies that are making a lot of profit off of essentially complete dependency on Ruby to work, I think that things like $5000 per month, there are a lot of companies for whom that is a blip on their budget and the cost of say, another hack of RubyGems.org or something of equal impact in the future cost so much more than that so if we’re looking at the tradeoffs and be thinking about the idea of what is a suitable amount to come in and that reflects the value to the company, it reflects the amount of dependency the company has. What is Ruby Together membership get you? I think, André take that one because we do have a variety of tradeoffs and benefits that we have as part of membership. ANDRÉ: Thanks, Carina. The benefits for individual devs at the $10-level is a warm and fuzzy feeling that you’re helping us. At the $40-level, you then become a full-on member of Ruby Together and what that means is we send you an invite to our private member’s only Slack and you then have a vote in the yearly election for the board of directors. The board of directors includes basically very community-minded Ruby developers who set our priorities for what we’re going to take our budget and try to accomplish. Our current members includes Steve Klabnik, Sarah Mei, Terence Lee, the director of the Ruby platform at Heroku. Coraline, who I hear that some of you have heard of and I guess, just joined the board, Camille Baldock. She works on the data team at Heroku. Really what we’re aiming for with this is to have representatives of individual agencies, representatives of fiercely individual communitarian mindset and representatives of the largest, most platform-y, kind of like use of Ruby. JESSICA: Of one thing that I like about this concept is that sometimes, when we talk about the community, we think only about individuals. But Ruby Together explicitly acknowledges that the community is a combination of companies and individuals. ANDRÉ: Absolutely. We’ve actually structured the company governance to reflect that and individual members and companies each get a vote in the board elections. While it’s very true that companies make more money and depend more heavily on this infrastructure for their businesses to function and profit, individuals have more say in what it is that we’re actually going to work on by virtue of there being more individuals that we have as members of the companies that we have as members and that was a deliberate choice that we made while we were setting up how the voting will work. JESSICA: I’m thinking to ask exactly what it is that Ruby Together supports in the interest of ensuring the future of Ruby. CARINA: The reason we want to fund these projects is right now, we need them to be stable. We’ve talked a bit about things like what happens when a person’s life changes. I mean, the people who are maintaining these projects now, eventually are going to have jobs that have more demands on their time or their family life is going to change or they just burn out on it for whatever reasons, people come and go. Part of Ruby Together’s work is to make sure that there are a new generation of people who are going to have the interest, and hopefully skills to be able to participate in these kinds of projects and have an interest in getting involved with them in the longer term. One of the things that we do with that money along with funding current developers is we’ve been involved in both Rails Girl Summer of Code and Google Summer of Code that provides funding and mentorship for, essentially newcomers to the field to work on practical open source projects for several months at a time and really get their hands on the code and really get to spend time with experienced developers as their mentors. André, you want to tell a little bit more about what that’s been for us? ANDRÉ: Yeah, absolutely. Ruby Together as it’s grown, we’ve kind of gone for the breadth over depth approach and this was another decision that we made early on, where we want the lottery winning number to be very high and we need a lot of people to win the lottery for all of Ruby infrastructure to collapse. Instead of hiring a single person to be a full time dev across all of these projects, what we’ve done is basically contracted out for five hours a week with a bunch of devs. I am one of the devs so I do work on Bundler and on Ruby Gems and sometimes on RubyGems.org. There’s David Radcliffe. His main job is at Shopify, doing ops work and he spends his Ruby Together time working on RubyGems.org. There’s Samuel Giddins. He’s a student at University of Chicago right now but he spends his time on Bundler and on Ruby Gems. He’s actually also on the CocoaPods team. He wrote the underlying dependency resolver that CocoaPods and Ruby Gems and Bundler all use to figure out which versions of packages can work together successfully. As we talked about in this month’s newsletter, we actually just hired Liz Abinante whose main jobs is at New Relic. She’s working on documentation. We gave her the title Empress of Documentation. As Bundler and Ruby Gems come closer together, we’re going to need more work to consolidate and integrate all of that scattered documentation that has accreted over the years. There’s Ellen Marie Dash, who’s been working on Ruby Gems and working on some of our internal tooling that we use to keep an eye on open source projects. I think, they’re working on called ‘how_is’. It’s basically like, “How is this open source project doing? What should we be looking at? What’s getting neglected? What’s getting taken care well?” I have really high hopes that that tooling, once it’s complete, will actually be super helpful for us as a team, as we’re trying to coordinate across all of these different projects and across all of these different people. To summarize, we have around five or six people doing around five hours of work per week, across the projects that we’re able to afford to pay for work on right now, which are Bundler, Ruby Gems like the gem command itself and RubyGems.org, the Rails app that runs all of the downloads made by both the gem command and by Bundler and gemstash, the server that we’ll probably talk more about later. JESSICA: Okay, so Ruby Together is able to pay a few people for their time. I love that because it’s so much more inclusive. There’s only a certain number of people who can give free time. ANDRÉ: That’s absolutely been one of our goals. Honestly, at this point I think I’m the only person who would be able to give time for free, if we were able to pay for it. As a college student, say I’m in definitely needs some kind of job to pay for things that he wants to take care of. He actually initially got involved in the project because of a grant from Stripe and that’s how he wrote the resolver that wound up going into CocoaPods and then into Bundler and then I — JESSICA: Is that part of Summer of Code? ANDRÉ: No, it was part of Stripe’s open source grants program, where they select a few people and give them some money to work on open source for a while. Then afterwards, I was able to say, “Ruby Together can actually pay you to continue helping with this,” and he said, “Oh, okay cool. I will keep doing that then.” JESSICA: Now, it’s time to a shout out for our supporters. Speaking of both companies and individuals are part of our community, the cost of this episode have been covered by Atomist. Atomist is my employer now and I’m super excited to be there because we are building developer automation tool such that every company can develop with smooth processes that save them time without having a whole team of full time people dedicated to developer tooling. Check it out and Atomist.com. CARINA: To me among other things, it’s an issue that I’ve been an advocate for quite a while which is fair pay. Usually, the contacts when we talk about fair pay is things like the gender pay gap. But the bottom line is that everyone deserves fair pay and this is a really common one to treat programmers as, “As an industry we’re well paid so stuff that you do for your own love of the community and the benefit the whole community somehow should be things that are done for free.” It’s a really unusual perspective that few other industries share the idea that incredibly valuable work that benefits everyone should be unlimited free labor is really not consistent frankly with capitalism, in general. Although, it is consistent with the idea of large companies capitalizing off of free labor so it’s somewhat consistent with capitalism. But overall, it also makes some big assumptions about individuals — not everyone in programming is making bundles of money off of it. Certainly, not everyone works for a startup. Not everyone is working in San Francisco. Lots of people are working at very different wages across the world so fair pay includes making sure that everyone’s work is valued for what it is. I think this is something that I really value about Ruby Together is that perspective that contributions of open source need to be fairly paid and that benefits everybody. That’s not just an ethical stance, it’s a really I think just sensible stance for the whole community to have. JAY: Amen. SAM: That raises a really interesting question, which is it can be really, really hard to get developers to pay for stuff like I make a six figure salary and when I see some bit of software that’s like $30, I’m like, “Do I really need that? I don’t know.” How do you overcome that developer inertia and that instinct to be like, “I just write the shit myself.” ANDRÉ: It’s been a really interesting challenge to be honest. The biggest thing that helps overcome the inertia is that programmers just feel an intense amount of positive things about Ruby and about gems and about sharing their code with other developers and getting to use code written by other developers. Honestly, I feel like the majority of response from individual devs has been, “Wow. Oh, my God. Holy shit. It cost that much to keep this up and going.” Actually, I really want that to stay up and keep going because I use it all the time and I care about it a lot. I guess that is important to me, it turns out as you mentioned it. That’s actually been really nice to get feedback from individual Ruby developers. But I guess, the flip side of that is that companies don’t have feelings and that makes it much harder to convince companies that they also want to give us large piles of money. CARINA: I’m going to make a distinction there because I think what’s interesting to me that I’ve observe is how much smaller companies, particularly agencies and consultancies have been much more willing to step up because my theory on this is that they’re much more attuned to the dollar value of an hour’s work so they notice any time they’re doing non-billable work and what the cost of that is. It’s much easier to look at something like Ruby Together and say, “Wow, you can save us money on not doing non-billable work and I know exactly what this cost the company and I’d be willing to pay some amount of that to not cost the company this. This is like a great idea.” Whereas, in large corporations, there’s a sort of like budget as a slush fund or everything has some sort of line item and as long as Ruby Together is perceived as a charity, rather than a business trade association that’s there for business purposes to serve profitability, I think it’s really hard to have that conversation with large companies because they don’t have a line item for charity. They don’t see it as a conversation to have. But Ruby Together is not a charity and I think that’s a major misconception that everyone can really help to dispel because when we go in to have those conversations with them, that’s the first thing that we have to overcome is this idea that it’s a donation, rather than it’s an investment. JAY: One of the things that you mentioned André is that once you said, “It cost this much,” then individual developers like, “Of course, we should support this.” But my question to you would be is do you think that we’re all for individual developers were all like, “We can help you offset this amount of money that’s going out the door but if you’re going to start making a profit off of this, we don’t want to support you in the amount of time spent on this.” We’re talking about server cost. That’s something completely different. Do you feel that it’s happening there? ANDRÉ: I feel like there’s a limited amount of that. I guess honestly at this point, I’ve been working on Bundler as an open source project for about six or seven years now. I mostly get the sense that developers are kind of mentally calculating that even if I keep doing this for another five years, I’m not going to ever have made enough money to pay for the hours that I spent over the last six or seven years. I think the equation ends up making it look like, “I’m still doing this out of caring about the community, rather than trying to make a buck,” and — JESSICA: It’s nice to have the excuse to tell your partner, though that this time is actually also bringing an income for the family. ANDRÉ: Absolutely true. Actually, that’s a pattern that we’ve seen play out multiple times in the people that we have working on open source. Our goal as an organization is to fund development on the open source infrastructure that the Ruby language needs and that makes it both incredibly easy to get people excited about supporting it because they have such positive feelings about the Ruby language and Ruby infrastructure. But also it means that when we are actually paying money to support that development work, suddenly people with small kids who would otherwise be giving up on donating their free time to keeping Ruby Gems up are like, “This actually helps me underwrite the new cost that just cropped up as a result of this kid and this is now extremely valuable to me to do and now RubyGems.org gets security patches applied on a regular basis, rather than on an as possible in our spare time around the requirements of the rest of my life with free work at the lowest priority.” This is played out for real in, I guess 2013, when there was a security issue. We knew about the security issue. All of us had it as soon as we have some time to do free work, we’ll fix it and none of us had time to do free work before a hacker figured out how to exploit it and Ruby Gems was down for like two weeks because we had to throw away the servers, we had to re-download every gem and check some of it to make sure it hadn’t been replaced with a Trojan. It just took so much time and effort to recover from that hack. I guess obviously, I can’t guarantee that we won’t get hacked now but I have high confidence that everything that we know about is fixed because we’re paying someone to fix everything that we know about as part of their high priority, “I’m getting paid time,” rather than as part of their lowest priority. This is fun but it’s just like throwing away free time for no good reason. JAY: That was such a great example. This is something I’m passionate about. I do a lot of training stuff here in Central Ohio. If I get another to developer who has tons of experience, who were working on some sort and [inaudible] something and they pull up Sublime Text and they just like, “I know I don’t want to pay for the license.” [inaudible] button and pops up some Sublime, like I would go crazy because dude, you’ve been in the industry for a while and what is this? This is something that I see like on a smaller scale but it happens so often like, “You know what? You should probably just go ahead pay for that,” in addition to having brought in a number of speakers locally and having to also fight for like, “Yeah, I don’t care. If Sandy Metz wants X or Michael Feathers wants X or whomever want X, like this person is making contributions to our community and making our community better,” I’ve definitely seen it where companies are like, “I don’t think so. Do they should just come by for free and they should get this infrastructure for free.” In you guys case, it is something I’ve seen quite a bit. It’s not really a question at all but it is one of those things that really kind of the ticks me off because I feel like once you hit a certain threshold, we have to really enable, I think people like you two to continue working on stuff because it’s meaningful and there’s a lot of people making money off of it. There’s that too. Come on. There’s another thing about it. Also, as a small business owner and someone who was more or less well-off financially, looking at software companies when you’re in this business that you had this huge profit margin and we pay a ton of money in taxes. If your company’s doing well, just ask what your tax bill was last year, unless you have some weird inversions thing going on. We have opportunities here to definitely support organizations like Ruby Together but for whatever reason, we just don’t. I’m sorry, I know that’s not a question but it’s something that really, really ticks me off. SAM: Totally. Hear, hear. CARINA: Thanks for the rant. ANDRÉ: Yeah, absolutely and it’s really is something that we’ve seen. My theory so far is that it’s just like the very strong status quo like programmers build these services for themselves and then companies expect to be able to use them because they’re there and they’re free because programmers want to make them free to other programmers. JESSICA: There is a good article that came out recently, retrospective on RethinkDB and it pointed out that the developer tools market is like a terrible market to be in. One reason is because people will do it for free because we’re developers and to be like developing things that we use so we’ll published those. The other reason is that reluctance of companies and developers to pay for things that there’s actually very little willingness to turn out money for this. This is just a personal perception because I now work for a company developing developer tools, which is awesome because I love developing tools for myself. ANDRÉ: Totally. I guess we now fall into that same bucket. It is actually pretty enjoyable to be able to fix things in Ruby Gems or in Bundler when there’s a problem. But I guess in a similar way, there are companies that I’ve talked to who felt like it was worth it to assign a full time salaried employee to work for six months, rather than give us a few hundred dollars and take advantage of some work that we had already done. JESSICA: Now, you’re going to be like the budget vagaries of I could pay a person because I’ve got that authorized and that comes out of this budget versus paying another company which would come out of a totally different budget then we get back to that bit about small companies are more willing, Carina I think you said. Sam said that companies don’t have feelings. Well, companies don’t make decisions, people do. In small companies, you’ve got individual people that looked like more power to make those decisions and they do have feelings and foresight, in some cases so that might contribute to a small company — [Laughter] ANDRÉ: — Foresight is a nice thing. We could use more company [inaudible] with foresight, I think. JESSICA: I think that might be a theme this season. SAM: If I can actually offer just a moment of hindsight as well, I’m old enough to remember what Ruby and Rails development was like before Bundler came along and — ANDRÉ: That’s very kind of you to remember. SAM: It was vendoring everything and if that didn’t work, hoping like heck that you could figure out the magical combination of all the version numbers of all the gems and plug ins that you depended on. Then RVM came along and made that slightly easier because you could use a gem set and then not have these gem versions interfering with those gem versions but still — ANDRÉ: It was less than ideal but I absolutely feel what you were just saying. In fact, I did an entire conference talk on just that. As a person who’s been doing Ruby since way back when and as a person who’s worked on Bundler, I basically took all of that historical experience and said, “I could actually give a conference talk about not just what exactly is Bundler doing anyway but why is it about? Why is it doing that specific thing?” So I have an entire conference talk that I gave at, it must have been RailsConf because it wasn’t the RubyConf that just happened. It’s called ‘How Does Bundler Work, Anyway?’ I have a blog post version of it as well. But it’s basically like how does it require work? How does loading things into the load path work? How does Ruby Gems work? How does Bundler work? And walking through like, what problems do you have when you’re here that leads you to the next solution and then what problems does the next solution create that lead you to the solution after that? It turns out and I didn’t even realize this at the time until I was working on the talk as background for Bundler but it’s almost like a blindingly, bright and clear path of like, “I have solved this problem and this has created a –” It takes me like 10 seconds of using the new solution to be like, “I really have this new problem,” and that walks you through 10 years of Ruby dev tools history in 30 minutes because like, “Actually, now that you have that, you definitely have this new problem that needs solving.” SAM: I guess, the short version of all of that and starting from my rant is like, “You kids don’t know how good you have it these days.” [Laughter] ANDRÉ: Maybe I should have included that at the end of my talk. SAM: Old man shout at cloud. JESSICA: I’m curious are there problems uncovered or problems created? Now, suddenly it’s easy to download things so you could get past that and you find the next problem. It’s easy to download things. Now, I’ve screwed myself by introducing thousands of dependencies. ANDRÉ: It’s absolutely both. I do talked about that a little bit in my talk but that is an interesting question and something that I didn’t focus on. Ultimately, I think you uncovered the next most painful problem and suddenly that’s your newest, most painful problem. At the same time by making that last generation problem go away, you’ve allowed yourself to do things that weren’t possible before that create new painful problems. JESSICA: Just because it was possible, it doesn’t mean a good idea. ANDRÉ: Yeah, that’s true. I guess I would certainly rather be in a world where there are thousands of gems than a world where there’s a little 10 gems because they’re so hard to install. CARINA: Okay, André, now give us the number of how many gems there actually are. ANDRÉ: Oh, okay. There’s around 120,000 gems now. That’s by name. It’s not every version. It’s just a gem with a name. There’s around 120,000 and then if you count every single version that’s ever been released of those 120,000 gems, we’re pushing about 1.2 million gems right now. It’s really a large number like it’s taken off in a big way and Ruby have felt very empowered to make gems. Definitely, I guess even related to that, I don’t talk about this in my talk at all but one of the things that Bundler did was ship with a standard library level way to create a new gem and a new gem skeleton and be able to run a rake release and suddenly it’s on RubyGems.org. We saw a really big spike in people creating their own gems because all they had to do was run Bundler gem and throw some code in a file and now they had a gem. JESSICA: What you make easy, people will do. ANDRÉ: Absolutely true. JESSICA: I was struck, André in your origin story. You said that Ruby was useless in 2004 because reuse was so hard. You had to go download somebody tarball off their website in order to reuse that code and people credit Rails with the thing that made the difference for Ruby, between being useless or not. Certainly, Rails was super important but without a package manager, without Bundler and Ruby Gems, it’s not the language, it’s the language system that we work in, including libraries like Rails. But the package manager is super important for exactly the reason you just enunciated that if you make it easy for people to share and build on each other work, they will. ANDRÉ: Absolutely and I honestly feel like that has historically been one of the biggest strengths of the entire Ruby community because even back when it was ridiculously hard to share code, people were still super enthusiastic about it. I guess at that point in the ecosystem that we had, considering how hard it was, people were just unreasonably excited about sharing and reusing Ruby code. I feel like that kind of like ethos has carried through both into the creation of Ruby Gems, which I think the very first beta of Ruby Gems launched in 2000 through 2004. Then it really started getting by and use around 2005. I remember the first Ruby conference I ever went to was in 2005 and I got to see these people show up and stare at them in total awe because they were on the Rails core team or they were creating Ruby Gems and be like, “Oh, my God. This is amazing. How did they get so smart and so capable and so accomplished?” This seems impossible as whatever I was at the time, a college junior. I feel like just that excitement of sharing things, really led to Ruby Gems, led to Bundler, led to the GitHub Ruby ecosystem that we have because Ruby is just excited about sharing code with each other. CARINA: We really are. I think going back to Jessica’s point about solving one set of problems allows us to essentially create whole new ones. The upside is that we’ve got those 1.2 million packages sitting on RubyGems.org. The downside of that is that the traffic for all of that is enormous. It was a year ago… Let’s see, I think it was 280 terabytes per month and that has continued to skyrocket. Ten months later, it was already 460 terabyte, which is about 60% growth in 10 months so you can see that already, the numbers are just enormous and they’re growing so much bigger. There’s this myth that essentially Ruby is in decline and when you look at those kind the numbers, not only as Ruby not in decline but our most troubling problem is that Ruby is still growing and growing and growing and there is financial cost to the community that right now is not being borne by the community for that. André, what is the server bill for RubyGems.org at this point? ANDRÉ: I guess to be clear, the server bill is currently covered by a combination of actual dollars from Ruby Together members, actual dollars from the Ruby central non-profit and donated services from Fastly. But if there were a single entity covering the entire server bill in one place, it would theoretically cost something like $40,000 a month. CARINA: Today? ANDRÉ: Right, that’s today with clearly increasing usage and expected higher costs over time. JAY: Do we think that this is like natural, organic interest in Ruby or is it something else? Like it’s individual kids and like, “I want to get into Ruby and I’m doing stuff,” or do we think this is more corporate usage. Any idea about that? ANDRÉ: I think it’s both. At this exact moment in our infrastructure life, it’s very hard to tell who exactly it is that’s asking us for gems because data centers don’t differentiate between people like we know that US east downloads a bajillion gems. But we don’t know who are the individual people that are renting computers for Amazon are. That’s something that we’re interested in trying to get better at figuring out but the overall story seems to be everyone is really excited about using a lot of Ruby. Even if people are talking about how Ruby is the newest, coolest stuff anymore at the moment, there are still a huge amount of Ruby developers, more than there ever have been before and they’re writing and deploying a huge amount of Ruby code more than there ever has been for. SAM: My sense is that there are a lot of companies for a while there, Rails was pretty much synonymous with a certain type of Silicon Valley startup. If you were doing a startup, you were hiring as many Rails developers as you could find. But there’s a spectrum from individual enthusiast just learning to program all the way up to a giant corporate entity using Rails and 50 gems. Only towards the very big end of that spectrum does it make sense for an organization to stand up its own private gem’s server so I would imagine that a whole lot of that traffic is coming from that middle section of companies that are big enough to have a couple of Rails developers but not big enough to have bothered spending the time setting up a private gem server. ANDRÉ: I guess, interestingly anecdotal evidence from employees at the very biggest end of companies is that mid-end companies as they grow are like, “We should stand up in an internal gem server,” and then as they grow again, they’re like, “It’s not worth the huge amount of operational costs to run own internal gem servers. Ruby gems actually does a really good job of running a worldwide infrastructure that allows all of our data centers fast access. Why don’t we just get it from them?” SAM: So never mind? ANDRÉ: I guess I would say not never mind. that is absolutely true and then there’s an additional life cycle step beyond that where they outgrow their own ops team and the ops team says, “It’s no longer worth it to spend our time on this. We could just make the RubyGems.org ops team take care of it. SAM: And not pay for it? JESSICA: For instance, Stripe totally has their own Ruby Gems server and the reason for that is security because you want control over that code that is getting to play out to all of your production side. If you had Ruby Gems, with this infrastructure to make it possible to download gems throughout the world, and also you had an organization backing it that had resources to make certain states secure, that would be ideal. ANDRÉ: That would be ideal and Stripe is basically the Ruby’s founding member of Ruby Together. They turned over money to the Bundler project when that actually meant writing André a personal check. I, then took the money from Stripe that they were giving me for no particular benefit to themselves. They didn’t put any strings on it. They just said like, “We use and like Bundler and if giving you money will help make it better, here’s some money.” I took that money and used it to pay for lawyers and pay for incorporation costs and get the IRS to give us non-profit status. Stripes, still to this day is by dollar amount, the highest contributor to Ruby Together on every single one. I, very much feel like Stripe as a company that recognizes how good it is for them to be supporting that. JAY: Good job, Stripe. Yay! CARINA: Now, that lead on Stripe for being the biggest contributor. Hey, every other company, wouldn’t you like to out-best them? Come on, you want to come and hire, right? JESSICA: Yeah, this should be at the top of their list. CARINA: Right, like we’re competitive industry. Come on y’all, come compete. JAY: I have a question about gemstash. At my place at work, we use geminabox, which meets our needs. What are some differences? Geminabox has been around for a while? ANDRÉ: Geminabox is a private gem server. You stand it up, you create private gems that you don’t want to push to public Ruby gems and then you put those in geminabox. Then you can add that geminabox server to your gem file as a source and say, “I want these gems to come from this source, geminabox over here.” That way, you can have your own gems that are internal but you get the versioning benefits, you get the speed of installation of a tarball versus a Git repo benefits and everyone can see the promises you’re making via version packages, rather than running alongside the bleeding edge Git repo. Gemstash is a product that Ruby Together funded the development as a direct response to talking to companies that we’re giving Ruby Together money. It’s both a private gem server like geminaboxes but it’s also a caching Ruby Gems mirror. This is really the thing that makes gemstash interesting because you can run gemstash on your laptop. Once you’ve installed a gem from Ruby Gems, you will never download it again. You can run gemstash in your office and tell Bundler that when it sees Ruby.Gems.org, it should instead look at your in-office mirror and now the 50 dev boxes in your office, only have to talk to Ruby Gems once and after that Bundler was installed as a local operation on the local network. JESSICA: Man, I need that for Elm. ANDRÉ: In retrospect, it’s such a good idea. Then like the last and big one is that if you have a data center and it’s very common to have like 50 or 100 separate apps servers in a data center, you can run a single gemstash in that data center and you just like solved the, “It’s really expensive to fetch the same gem a hundred times from the internet problem,” without having to check a giant tar or gzip or blob into your Git repo, which has been the traditional solution for that. It’s miserable. Gemstash, I guess is both an effort for us to support that private gem infrastructure in a way that looks just like Ruby Gems so like gemstash supports gem push and gem yank, where geminabox is like written its own client infrastructure so you use a separate geminabox command to push a private gem. The way gemstash is written. It uses the same public API as RubyGems.org so you say gem push to this server and now you have it in your gemstash. Then I was saying that secondary factor of like a pass through cache/mirror of RubyGems.org is a really powerful thing under certain circumstances. That’s why we built gemstash to address that pain point that we heard from companies where, honestly kind of like big companies. We talked to them and every single one of them said, “Yeah, we’ve assigned a full time employee for months to solve this local internal gem server/Ruby Gems mirror problem,” and so we said, “Well, we can solve that better than you and we can solve it in less work than you because we’re already familiar with the problem.” CARINA: I think a really concrete example that we can look to in the past is Heartbleed and OpenSSL. Before Heartbleed happened, OpenSSL was getting something like about $2000 per year of total contributions to support a handful of developers to work on OpenSSL. Now, the entire industry depends on OpenSSL and nobody was thinking about it at all. Clearly, not financially but in general so much infrastructure that we just take for granted. The moment we stopped taking it for granted was when Heartbleed happened and suddenly, every company had to scramble and the amount of lost time, the amount of I think in many cases was lost goodwill. You know, customers don’t always understand why something isn’t working. They may just look at your company and say, “Well, this product isn’t working. Your server is down. Clearly, your company is terrible. Why should I do business with you?” There was all this cost of doing support, there was developer time. Altogether, the industry lost over a half a billion dollars on just cleaning up after Heartbleed. Now, you look at that kind of thing and you say, “I don’t know what the number would be if a similar situation happened within the Ruby community but you can’t send an individual company basis, what would be the cost of a major security breach again of something like RubyGems.org. What was the cost when it happened last time? You can look at those and come up with some real world numbers of what is the value to our company in making sure that these projects are stable, that they’re secure, that they’re understanding community wide problems and addressing them? It matters. JESSICA: Compare that to the cost of a monthly membership in Ruby Together and Ruby Together looks like a really good idea along with SSL together and who knows what else. It also gets to the point that you like to think that software is [inaudible] once used it forever. But very few programs are like that. Software is an ongoing expense, just using software is an ongoing expense. It is not a capital expenditure. ANDRÉ: Yeah, definitely. One of the big outcomes of Heartbleed was that people suddenly cared enough about OpenSSL to donate money to the project. There are very succinct and clear write ups of what the outcomes and results from that are. Heartbleed led to enough donations to cover the cost of working on OpenSSL for exactly one year. That year is now over. Open SSL no longer has funds to sustain ongoing development. If there is another Heartbleed hiding inside OpenSSL somewhere that no one has found yet, it won’t get fixed until it’s out in the wild because no one has the money to actively work on it right now. Seeing that happen basically served as a giant wake up call for me to say, “This needs to be not something where I go to a company and say, please give me a pile of money and I’ll go away.” This needs to be something where I go to a company and say, “This is a danger to you and your company and a huge potentially cost every single day that you’re in business and we need to keep preventing it from costing you money, basically as long as this matters to you.” CARINA: Yeah, the question I get asked on a fairly regular basis is why are you doing things this way. It seems like it’s hard. Why don’t you just go to a couple of companies and get a big fat grant? The answer, first of all is that there aren’t that many companies that are offering big fat grants but a bigger problem is just sustainability. You’re going to need to go back next year and hope that again you can get fat grant from them. We need a base of ongoing contributions, not a one-time infusion. In the Django community, they had one of those one-time infusions, they used to pay a developer to work on Django, the money ran out after a couple of months and then what? You got someone who was working full time, who has taken time out of their life and career. When the money runs out, what is the project do? What is the individual do? We have to be able to make promises that developer can depend on. As André said, we’re paying for about five hours per week to various developers. When you can make a promise that that’s going to happen every week, then they can also make an arrangement with their employers to say this is the promise I’m making to you too. I’m only available 35 hours a week for this job or whatever it is because I’ve made a commitment to five hours a week somewhere else. It’s about making things really logistically possible as well. When we make a commitment to developers, they’re able to make a commitment to their family and their employers so one-time versus sustainability matters for a lot of different reasons. JESSICA: For subscription model totally works for software and when you have community infrastructure that’s helping everyone, then your subscription can be very small as part of a large group of subscriptions. But it’s still if you’re not paying for software maintenance, you’re not getting maintained software. ANDRÉ: Absolutely and that’s really what we’re hoping for is that a large enough number of people will be able to chip in and a large enough number of companies will be able to chip in that we cannot just keep this from falling over but actively make it better for everyone who uses Ruby. JESSICA: Yeah and we talked earlier about how one reason, it’s hard to get people to pay for development tools is that developers will develop them for free. But that’s not sustainable because either, they become abandonware as the developers or developing them are done with developing them — I want to say the word development several more times in the sentence — SAM: Thank you, Steve Ballmer. JESSICA: — Or that tool that I developed that I was so proud of becomes a burden to the individual until I choose to give it up so yeah, buy your Sublime license and ask your company to join Ruby Together. CARINA: I think this is an interesting example because some things are harder than others to make. When somebody looks at that license fee for Sublime and says, “I don’t want to pay for it.” Okay, well you have other options. You can use lots of free other options like say, Vim or Emacs. The person’s answer is usually is, “Oh, that’s hard. It would take me time to learn to use that. This is so much faster and easier.” Well, yes it takes time. What is your time worth? It’s probably definitely worth the cost of that licensing fee. Similarly, you could in theory make another Bundler. How much time and expertise do you think it would take to be a person who can create another Bundler and maintain it? These are hard niche problems and there’s only a few people in the Ruby community who have currently the expertise to be able to pull that off. It makes so much more sense to pay people to do stuff that they’re good at, rather than have so many people inside companies kind of chipping away at problems with this, that they don’t know as well how to solve. They’re spending a lot of time and stuff that isn’t the company’s core competence, their core product and they can’t do it nearly as cost effectively as people who are working on these projects already. It’s just silly the way that we’re approaching things right now as an industry. What is the value of your time invested in making a semi-good-ish result? It doesn’t make sense. ANDRÉ: Absolutely. SAM: It’s time for a listener shout out. Today’s shout out goes to Polly Schandorf. Polly is @N3rdyTeacher — that’s nerdy with a three — on Twitter and as you might guess from that, she’s a former school teacher who has made the switch to a programming career. I paired with Polly at Ruby Geek Camp a few months ago and I can personally attest that she is indeed a very cool person. Thanks very much for your support, Polly. We want this to be a 100% listener-supported show. If you feel inspired to help us out like Polly does, head over to Patreon.com/GreaterThanCode. Any donation will get you into our supporters only Slack and we may have some other perks as well that you can see on our Patreon page but any amount is definitely appreciated. Thanks, listeners. We would like to end the show with a period of short reflection and that can be a call to action or something you’re going to take away from this or something that really surprised you and it’s going to stick with you on the way home or whatever. There’s been so much that we’ve talked about today that has my brain buzzing. It’s hard to pick one thing so I’ll go with the easy route, which is that during this call, I have officially become a friend of Ruby Together and I’m donating now. Thank you for reminding me to get off my butt and actually do that because the stuff is really important and thank you for your work. ANDRÉ: Awesome. Thank you, Sam. JESSICA: Sam, that’s awesome, except it’s not donating — SAM: Damn it! JESSICA: — You’ve joined Ruby Together. SAM: I am a member friend or something. JESSICA: Before this podcast, I didn’t know what Ruby Together was. Based on the name, I thought it was going to be another warm, fuzzy Ruby thing but it’s much more important than that. This is an advancement in the software industry as a whole to form a trade organization that is a business, related to supporting all businesses and people and making our software infrastructure sustainable. As in the Heartbleed and OpenSSL example, this is something that many, many more software communities desperately need. I hope that Ruby Together is the front wave of an innovation that gives us a sustainable software infrastructure even more widely than Ruby. ANDRÉ: I would love to see that happen. That’s definitely kind of the underlying idea is hopefully that this model can apply to other situations and I know that there are some echoes of this kind of situation happening in the Python community. Although, I’m much less familiar with it than I am with the Ruby community. While I am only prepared to make this this kind of situation happen for Ruby, I really hope that other developers and other communities will try to figure out if this is something that they can make sustainable as well. CARINA: My take away today is that OpenSSL is back to being insufficiently funded. I did not know that and that scares the pants off of me. Ruby Together is an important project for this community — for Ruby community — but there’s so many other projects need something supporting them too. There’s things like the Linux Foundation. There are so many different models out there for how to do that but there’s so many projects that have insufficient or no funding at all and we’re all relying on them. That scares me a lot. I think it’s part of the reason why I care so much about Ruby Together is because these kinds of things have to exist. We just can’t be gambling like this. That’s really frightening to me. I will say that there is a great report on that bigger picture called ‘Roads and Bridges: The Unseen Labor Behind Our Digital Infrastructure’. It was written for the Ford Foundation by Nadia Eghbal who has been doing a great series of talks, drawing on the knowledge that she developed for this. It’s a real long read and it’s a really worthwhile read that gives a great insight into just how many problems we have with depending on open source infrastructure. Not because the dependency is bad but because it’s the unpaid open source infrastructure that we’re so reliant on and where that shifts the burden that companies are making on all of these projects. I really recommend reading or at least, skimming it and seeing some of Nadia’s talk where the most recent was a keynote at linux.conf.au just a week or so ago. ANDRÉ: I’m going to take the easy way out and say that my reflection is that I really hope that this is very convincing and I really hope that the devs and companies that listen to the show and care about Ruby will join as members and will help us keep doing this work to actually keep all of Ruby Gems and all of the code-sharing ecosystem that is such a big part of what makes Ruby great continuing to work into the future. CARINA: Join Ruby Together. Tell your manager why your company should join Ruby Together and tell them why it should be at an aggressively high level that really reflects the value and importance and profitability of their dependence on Ruby and Ruby infrastructure projects so come in and come in at a level that really can help us to do important work that you think is worthwhile and know is necessary. SAM: Yes, please. Thank you, everybody. This has been a really informative and illuminating show, as always. It was wonderful to talk to you all and listeners, we will be back at you next time.