Wicked Good Development Episode 8: Devnexus reflections and conversations Part 2

Wicked Good Development is dedicated to the future of open source. This space is to learn about the latest in the developer community and talk shop with open source software innovators and experts in the industry.

Brad Wood, Software Architect at Ortus Solutions says "everybody should be secure" when it comes to open source security. Hear Brad's distinctive perspective after occupying the roles of both an open source maintainer and contributor. From the advantages of using ColdFusion to the effects of trust in development like abandonware and namespace confusion attacks, Brad covers it all with Wicked Good Development at Devnexus 2022.

If you'd like to start at the beginning, jump back to part one of this series or move ahead to part three.

Listen to the episode

Wicked Good Development is available wherever you find your podcasts. Visit our page on Spotify's anchor.fm

Show notes

Hosts

• Kadi Grigg
• Omar Torres

Panelists

• Rohan Bhaumik, Product Manager

• Theresa Mammarella, Developer Advocate

• Steve Poole, Developer Advocate

• Rishav Mishra, Product Manager

• A.J. Brown, Principal Engineer

• Brad Wood - Software Architect and Platform Evangelist at Ortus Solutions, Corp

Transcript

Kadi
Rohan, you also had a conversation with not only a contributor about a maintainer. What were your thoughts on the conversation with Brad Wood?

Rohan
So he provided a perspective of what open-source maintainers, as well as contributors, look for. And there's an interesting nuance between the needs of those two groups that came out, that I wasn't necessarily clued into. So maintainers depend on contributions from different projects from different people in the world to keep the project relevant.

Contributors also take projects by means of their contributions to completely uncharted territory. And I think maintainers find that really satisfying from a creative standpoint. What I mean by that is, “Hey, this cool thing that I created for this one specific use case.” Contributors like adding to it and making it relevant for a whole plethora of new use cases that I didn't even think about. Isn't that cool?

So as much as possible, my key takeaway was that anything we can do to allow people to continue to do that or make it easier for maintainers to, you know, allow people to contribute easily. That is something that I'm finding that is something that I want to aspire to. So for me, that was very educational.

Rohan
Well, this will be a fun podcast. So Brad welcome. Can you help us by talking us through what it is you do and why you like being here at Devnexus? What have you thought about the entire thing?

Brad
What makes you think I like being here?

Rohan
You are here, aren’t you?

Brad
I am here. Right? So my name is Brad Wood and I'm a… I don't know what I am…. a software architect at a company called Ortus Solutions and we offer consulting and we build open source libraries for ColdFusion developers, which is always fun because usually when I say that people say things like, “Oh, I didn't know ColdFusion was still in use. Is that still a thing?”

Yes, yes, it is. It's a JVM language and it has been since the early 2000s.

So that leads me to your next question, which is what I'm doing at Devnexus. So the Java ecosystem is our ecosystem to a degree. And the JVM is where we live and where we execute. So if it runs, if Java runs on it, then ColdFusion runs on it and that's why I have websites running on raspberry pi’s because Java runs in the raspberry pi.

Rohan
Nice.

Brad
So yeah, we show up at Devnexus and we see what Java is up to. We see what the tooling is up to. We see how people are, you know, securing things like artifact repositories and all that good stuff. And that's what we do. So there's open source and free ColdFusion engines out there as well as Adobe ColdFusion.

And so we kind of sit in that space and we provide, you know, tooling and command line stuff and NVC frameworks. There you go.

Rohan
Awesome. So for those of us who were probably not as familiar with ColdFusion, as we should be, can you talk us to talk us through some practical scenarios where working with ColdFusion on a JVM would be advantageous and why?

Brad
Yeah, absolutely. So ColdFusion came out in 1995 which was a popular year for programming languages. That's when Java came out, Javascript came out, PHP came out, Ruby came out.

All of these languages appeared in the mid-nineties and ColdFusion started out as a tag-based language that, you know, the great benefit and downfall was like anybody, whether or not they knew how to program could just cram these CF tags in the middle of their HTML and they could prototype websites. And it was really great.

Obviously, all those languages I mentioned, it's evolved a ton. You know in the meantime, now it's a scripting language. It has a lot of functional programming enclosures and its main allure is just the amount of productivity it provides out of the box. So falling is all rim there's hibernate built-in.

If we want to create PDFs, I can create PDFs. I can integrate with soap. I don't know if I'd want to, but I can. I can build rest web services. I can, you know, work with archives. I can work with FTP, SMTP, POP3, image manipulation. You know, there's just a bunch of stuff that comes out of the box. And of course, most of that's all powered by open source Java libraries. You see ColdFusion as a giant collection of Java libraries stitched together on top of this JVM language.

It's a dynamic language and it's loosely typed. So it borrows a lot more from like Javascript or even Ruby. And it was fun to program in because there's a little code, or less code than sometimes statically typed languages. You can easily, you know, prototype stuff is just-in-time compiled.

So there's no compilation step. You can just edit your photo, your files on production. Not that you should do that mind you, but you know, if it's possible.

Rohan
Yeah. It is possible.

Brad
You know, you're developing workflow can be a very tight loop. You know, I can run a test watcher in my console and I can just, you know, be right in my test as I'm writing code and saving files and nothing needs to be compiled.

So it's a fun language to write in and it's very productive out of the box. I can do, you know, a lot of stuff with it.

Rohan
Awesome. So you mentioned that there's a strong open source contributor component to being successful with CloudFusion as well. And you also touched a little bit on security and related challenges, and I think you picked up on that at Devnexus as well. So what are your thoughts on open source security where as it pertains to the CF space to put it?

Brad
That's a great question. I think everybody should be secure. That's my take on it.

Rohan
Laughs Mic drop.

Brad
A lot of ColdFusion clients just historically, as a kind of an enterprise-focused product, are government, their corporate, their enterprise, and as a result, a lot of them have a strong focus on security. Which I think is a fantastic thing.

It's a smaller space right now, and there's not as many libraries as you know, like node and there are like eleventy billion packages. So that's sort of a blessing in disguise in that being behind the ball on some of the open source progress in ColdFusion, we haven't hit as many of the problems with malicious packages on our package repositories.

But I mean, there's a lot of tooling that's out there and our company tries to work with that. As far as stuff like there's a service called HackMyCF and it will literally, you know — you give it permission obviously — but, you know, it'll scan your, your web servers once a week or whatever, and will look for vulnerable paths in the URLs or any kind of exploits that appear to exist and then we'll email you right away. Or say, you know, JVMs out of date, things like that.

There's also some tools like Fixinator made by a company called Foundeo, and they will scan through your package dependencies and scan through your code and look for out-of-date vulnerable libraries you might use. But there's a lot of uncovered endpoints.

You know, forgebox.io is a website that my company manages, which is sort of a package repository for modules. And, you know, it has the same potential downsides that Maven does or NPM does in that.

Anyone can sign up for an account, they can upload a package. And we know we heard it in the keynote this morning, that package may be malicious. And if you can trick someone, you know, via clever typo and installing the malicious package you have to have a lot of, um, uh, what's the word I'm looking for?

The developers making those choices have to be very conscious of what they're choosing and what they're installing. Is it coming from a maintainer that they trust or is it just a random package? And so a lot of those kinds of problems, I think, are just endemic to package management and artifact management in general, regardless of language.

Rohan
So that's interesting because at Sonatype we essentially helped build and still maintain Maven central. And, you're right. These are some of the problems that we encounter. You're right that we have made it incredibly easy for people — obviously, they go through some checks and balances — but essentially anybody can, you know, share code with the world and host it in package managers.

You spent a lot of time obviously thinking about it. You said you work with a company that has their own package manager. So what can package managers do to guard against just people putting up packages out there? Is there anything that can be done while balancing ensuring that people can still have the freedom to upload to contribute, but maybe with some checks and balances? Or is that like the slippery slope? Overloaded question, I know.

Brad
Right, right. No, I mean, I understand because you want to have a community in which people can freely contribute things, but you know, within that is the ability for people to freely contribute malicious things. Obviously, you guys are from Sonatype and I was at a talk this morning, it was one of your colleagues and they talked about some of the things that Sonatype was doing, and we're going to be looking at implementing some of the same things. We’ve had exploits such as, um, I forget the fancy name he had for it….but if you have a private package, this namespace and you create a public namespace…

Kadi
Namespace confusion.

Brad
Yes, namespace confusion — you create a public package that, you know, just doesn't have the namespace and it looks like the private package as exactly how our package repository works. And so there's some heuristics that we should be able to apply that looks for a public package that might be masquerading as a private package. Also doing, you know, string comparisons, looking for packages with very similar names where they've just changed one letter.

We can at least try to create some reports that would alert us. Though, I think it's difficult to remove the human element from that. I would hate to just nix a package in an automated fashion, which could totally screw up someone's workflow, but we could at least create, you know, an internal red flag that says, “look at this package, look at this user,” because we already have people automating accounts on our online repository with the goal of just putting it up, you know, taking advantage of our public profile page that lists like the link to their bio or their Twitter.

But then, you know, it's like an online pharmacy link, right? And the whole goal was just to use our website, to get those, those links, um, you know, indexed. And it's like, how do we have a site that allows people to, without restrictions, join a community and still find that stuff other than just trying to find it and flag it.

Now there is some nice things we've also looked at doing such as having trusted package owners, perhaps, where certain larger shops or people that do a lot of packages could go through some process and we could say, “This is to some degree, a trusted author.”

I mean, it's tricky because I don't want to be held legally liable for all the codes someone writes, but, it's like, it's like package signing now. How can we have a notification or a designation that says, “This is someone that at least we're familiar with and we trust, and this package belongs to them.”

Also, we've looked at doing code metrics or even vulnerability scanning as packages are uploaded to be able to possibly market as safe or market as scanned. It's not always easy to do those sorts of things. And it's tricky because a lot of artifact repositories can have the problem where the binary and the repository might not match the code that's on GitHub.

And so if you pull up a package and you go look at their GitHub repo, you may scan through the code and think this all looks legit, but is that code the same as what's compiled in the actual binary or downloading, unless you actually look at that source. So it's, it's a tricky thing to work with as obviously you're aware.

Rohan
Yeah. These are problems that I think, so I think what I'm hearing is there is a balance of like trusted authority but also trying to guard what is actually published with the techniques that you talked about. There'll be the tone of that currently is using around electrical jacket, I think.

So when I asked the question, I was trying to see what other package managers are trying to do in this space. And I think the more I think about it, the more I talk to people, there's like a blended solution. There's a blended approach to solve it. Like, I don't think getting access to package managers from only two only trusted people is the way forward. Like I think that is an anti-goal.

Switching gears, when we were chatting earlier, we were talking about just generally open source development, like you, you are and have been a maintainer and contributed to projects. I wanted to get your sense on the motivations or why do people actually decide to take time out from their day job, or take time that could be spent with family or doing other things. And why build software for other people that are part of the community? What draws you to it?

Brad
Yeah, that's an excellent question. Kind of in the late nineties, when we sort of had the rise of Linux and the new project, into the idea of this massive piece of software, that thousands of people around the world just devote their free time to and to give it away.

A lot of people were like, that's ridiculous. Why would anybody ever do that? I can think of two reasons, two big reasons off the top of my head.

The first is more of a practical one, which is, you know, I'm using a library and I'm integrating it with my own software and it's open source. And so it doesn't cost me anything, but I hit a bug.

Right. And then the maintainer, maybe doesn't have time to fix it right then or I need to fix that ASAP. You know, I had the ability to say, “Well, this is important to me. And this is important to the project I'm working on. I can go and fork this project and I can fix this bug and I can get my release out the door and then I can contribute that back because I've already done the work.” So it was sort of a practicality standpoint in that it can help remove roadblocks from me.

But I'd hate, I'd be remiss if I didn't think there's also a larger, or more, I don't want to say psychological, but there's something about being part of a greater community and a greater good that I think also resonates with people and that they want to work on projects that don't just benefit them, but projects that benefits a community of people.

And it's very rewarding. You know, obviously, it takes a lot of time, but it's very rewarding when you build something and you put it out there and other people use it. And so as an open source maintainer, one of the coolest things is not only when I get contributions from other people, but when I talk to people to say, “Oh yeah, I use your tool and I'm doing the stuff here” and I'm like, I didn't even think about that. That's a really cool use case. It's a really rewarding experience to be part of something that's bigger than yourself in a community. And you can get that in open source. And it's very unstructured. It's not under the guise necessarily of a particular company or even a particular goal.

It's just a group of people whose commonality is they like to solve problems. They like to contribute. I think that's a really great motivator for a lot of people that just want to be a part of something. Even if it may, it might not directly, you know, benefit them that minute.

Kadi
I think it's also a part of paying it forward. Right. Like you're using someone else's open source. So now you get to pay that forward too for someone else to use.

Brad
Yeah, absolutely. I mean, there's software that I use and I don't pay him any money and that benefits me greatly. So I think it's very fair that I pay that forward to other people, you know the same way otherwise, I could just be a consumer. But you know, the whole ecosystem doesn't work if everybody's the consumer. There has to be people that are contributing for sure.

Rohan
So during your time as a maintainer, I heard you talk about why you would continue to do it, but I guess what were the things that you were doing as a maintainer that made it hard for you to keep doing it?

Or what took most of your time? If you can provide some insight on that?

Brad
Hmm… I don't know. That's a good question. I've never quite thought of it that way.

There can be a struggle oftentimes, cause I mean, you see a lot of projects and I talked about in my talk this morning, some of the downsides of open source can be, you get a lot of abandonware, and even the Java ecosystem, as long as it's been around has a lot of that.

There's a lot of libraries that someone creates. Some people use it and you know, the maintainer can move on. They don't necessarily want to spend a lot of time on it any longer. And that can be difficult for open source adoption because you want to make sure you'll have support on things in the future.

One of the things that can be hard, I think as a maintainer, is making sure that you continue to have time. Especially if we have a lot of little libraries out there and when people do contribute, you can't always just say, “Thanks,” and hit merge. If it's a typo on my documentation, which I get those all the time because I type all the time, you know, that can be a no-brainer.

But you know, if someone's thought through a new feature or kind of a complex bug fix, I have to take time to sit down and kind of review what they've done and go through and make sure it doesn't break anything and run the test on it.

I try to always be really aware that I'm not wasting other people's time because I know it's really frustrating to try to contribute to open source because I've been there and you put in hours and then the maintainer just kind of doesn't really look at it and says, “Yeah, I'm really busy. I don't have time to mess with this.”

Kadi
That’s got to be a little heartbreaking for the developer.

Brad
It's frustrating. And you know, I've found through experience that people keep trying for very long on the same project. You know after a few times of spending hours and sending a pull request, if that's just being ignored by the maintainers, they're going to go elsewhere. And that's, a potential that…… co-maintainers, do you had to help with a project that are like, “yeah, I'm out. I'm not going to waste my time.”

So as a maintainer of projects, I try to be really, you know, careful not to be that guy, not to be the someone who's turning others off from open source. And so what I hate is if I pull up a repo and I look and, “I'm like, oh my gosh, I had a pull request in here from last month.” and I missed the email, I feel like a jerk now. I hope these guys, you know, don't hate me. I was traveling. I was busy. Because you're kind of signing up, implicitly when you do that, you know, to also be on the hook to be, you know, fixing things. I feel responsible for it, you know, and not everybody does, sometimes they just lead the project and go, “Yeah, I don't have time for it.”

But you know, I think to be a good maintainer, you're kind of signing up. You know, a future of responsibility paying attention and helping coach people when they send a pull request and you're like, “Eh, that's not quite how I would do it.” Let me give them some feedback and help them.

My rule of thumb is… my goal is to merge something. Even if it's not exactly the way I would write it, if there's, if there's value in what they sent and maybe they didn't have the time to keep going round and around, okay, I'm gonna merge what they did. So we have a success there and then I can go in and I can clean it up a bit, a bit more on my end. You know that way that person feels like they contributed and they did.

And then I can still go back and maybe run a code format or if it wasn't all spaced the same. But there's just a lot of time that was taken in that and then I have to go cut a release so that project to make sure they can use it.

And I think that is a difficulty for a lot of maintainers that maybe don't set aside the time for that. I always feel like that's kind of a big responsibility and I don't want to be the guy who's the jerk maintainer that put a project out there and isn't maintaining it. Not to judge anyone who has. I mean I'm not in a position to judge someone, who's put their stuff out there for free. And that's the beauty of open source. You see a project that someone doesn't have the time to maintain and then, you know, the community can take a fork of that and they can run with it.

Rohan
I think I found one thing that you said really interesting, which was around the fact that “Hey, if someone maintains something that isn't necessarily, so if someone contributes something that isn't necessarily formatted the way you expect? but it does, and it passes tests, for example, like you would merge that in and then figure out what to do with it.

I was speaking with, Sheng Wu, who's a director at Apache and he runs this Apache SkyWalking project and what he said was really interesting, which echoes what you said, which was around, they have contributors from all over the world. And I'm paraphrasing here, but like he's not in the business of dictating how people structure or write their code as long as it adheres to some loose guidelines and it makes sense and it runs.

So like, I think there are patterns there that I find really, really interesting.

You touched briefly around the frustrations of what you imagined a contributor could go through and that kind of directs how you view your role as a maintainer. As a contributor and you've, I'm pretty sure you've been a contributor yourself, and I expect that you can continue to contribute to open source projects that are outside of you own, what are some of the things that turn you off when you're trying to contribute to an open source project, besides the maintainer not being responsive?

Brad
Most of the frustrations usually revolve around an unresponsive maintainer. Sometimes if the maintainers a super picky- a buddy of mine was submitting a pull request recently to a website to add support for ColdFusion.

And, you know, you put in a curl command and we'll translate it to a language. And he was messaging me cause he was really frustrated because the maintainer were responsive, but the maintainer was incredibly picky and kept giving him, “Could you change this?” This will piecemeal changes like 40 of them in a row, you know?

And he was like, “oh my gosh, I don't have all day to fix every single spacing item. I mean, I think that kinda goes back to that. I try to think beggars can't be choosers as far as people spent their time to contribute and let’s see what I can make of it. You know, it's frustrating if you send a contribution and then it says for a while and they come back and they're like, “Well now there's merge conflicts, you know, can you fix those?”

And I'm thinking, well, there weren't merged conflicts when I submitted it, but you waited six months to look at it. As long as I can get feedback and I can get something merged I'm usually happy to be able to contribute. The trick oftentimes is just being able to get the attention of who's involved and get feedback on it.

I mean, I have opened a pull request for projects like JBoss undertow right now. And I've still been waiting since December to try to get feedback on that because of the bug fixes that I need and it's holding up my own releases.

Rohan
So what do you do with that? Fork and trust….

Brad
No, I just bugged the maintainers. I emailed them and I'm like, “Hey guys, how's it going? Can you please look at my pull request?”

So one of the things I talked about today is when you're going to contribute, stalk the community a little bit and say, “Do they have a ticket tracker? Right? Do they have a slack team? Do they have a discourse? Do they have a mailing list?”

So I mentioned, I have open pull requests with JBOSS undertow and I'm on their mailing list. Right. I know where their ticket tracker is. They have a hosted JIRA on-demand instance. And soI can use those tools to politely follow up if it's been a month and there's been nothing. I can reach out and ping them on the mailing list and say, “Hey, I'd love to get feedback on this.”

I think the important thing is to remember they're busy and they have a lot of things they're doing. And so politely begging a little bit for them to maybe give some attention, but wait a week or a month and give them an opportunity to look at it. Then ping them and say, “Hey, I'd love to get your guys' feedback on that.”

Usually, you can get a lot of mileage with that because most maintainers do want your contributions and they do want to help. They're just incredibly busy and you have to be able to make that work.

Rohan
Well, that's interesting.

I'm actually at a point, like, I don't know if I have any more questions do you, Kadi.

Kadi
I don't right now.

Rohan
I ran through everything that was in my head.

Kadi
I'm still, honestly, just stuck on the fact that the ColdFusion just because I don't see it that often. I just find that interesting.

Brad
So 70% of Fortune 100 companies use ColdFusion somewhere in their stack.

Kadi
I don't hear it that often so that's wild to me when you think of those stats like that.

Brad
Part of that is probably because you don't hang out in the ColdFusion communities. I mean, I don't hear much about Ruby, but I don't go out of my way to hang out in Ruby communities. I mean, there's a public ColdFusion slack team with 3000 people.

Adobe does a conference every year that usually has over 500 people at it. There's user groups around the world. There's conferences in Germany that I go to. My company has a conference in Houston. There was a conference in Vegas. I mean, there's ColdFusion conferences that happen.

There's a lot of activity, but if you're not listening to it, I joke, even though it's sad, that the only mainstream media coverage that ColdFusion tends to get is when a vulnerability will come out and there's a patch. And every software has vulnerabilities. I have blog posts that show how ColdFusion has fewer vulnerabilities than Java or fewer vulnerabilities in PHP, you know, and .NET.

I mean as far as CVE counts, it's no worse than other major technologies. But if the only news people ever hear is on the news, that one time a year, there's a CVE that's fixed. Then you know, it's like all you ever hear like, “Ah, ColdFusion is so insecure!” I'm like, oh, here we go again.

But yeah, that's the biggest problem ColdFusion has, in my opinion, isan image problem. People just legitimately didn't realize it's still out there into some modern JVM scripting language that people are using for real-world problems. And, but I mean, if anyone's listening and they're like,” I did ColdFusion20 years ago.” I mean, I've run into like six people at the conference this week that are like, “Yeah, I did ColdFusion a long time ago.”

I mean, if anybody wants to look into it again, look into Lucy server, which is the free open source engine. There's no licensing attached to that and look into CommandBox CLI, which is the tool that I'm the lead developer of. And it's a CLI tool written in ColdFusion that runs natively on Mac, Linux and Windows.

You can spin up servers that integrates with the adoptium API. So you can just say, give me Java 11, update, whatever, you know, give me Adobe ColdFusion 2021 update 3, it'll download whatever you want. It's a package manager as a rappel, you can run, you know, command line script and the ColdFusion things.

Most people didn't even realize it existed and all of that's open source. And if you'd mentioned ColdFusion on Twitter, I'll probably pop up like Clippy, the Microsoft Clippy and say something. I'm the guy. Usually, if someone's making fun of ColdFusion, I will inject myself into the conversation and try to reason with them.

Um, and try to reason with them. Uh, yeah, I'm, I'm not on Twitter to a few, people's that ColdFusion guy that always replies when they're ….

Kadi
I was going to say, can we please make that your Twitter handle now @ThatColdFusionGuy.

Brad
I mean, it's something I'm passionate about. You know, I like people that are passionate about a language or passionate about our technology and I don't just use it. I want to take over the world with it, you know?

And I think that people who aren't passionate just don't care as much. Definitely check out, like I said, CommandBox CLI and Lucy server and play around with it. It's really a fun language. One of my goals in life is to increase awareness of it because JVM languages I think are really cool.

There's so much benefit to being on the JVM. The scalability, the ability to run almost anywhere, the ecosystem of it, that’s really what I really love about it.

@ThatColdFusionGuy.

Brad
I mean, it's something I'm passionate about. You know, I like people that are passionate about a language or passionate about our technology and I don't just use it. I want to take over the world with it, you know?

And I think that people who aren't passionate just don't care as much. Definitely check out, like I said, CommandBox CLI and Lucy server and play around with it. It's really a fun language. One of my goals in life is to increase awareness of it because JVM languages I think are really cool.

There's so much benefit to being on the JVM. The scalability, the ability to run almost anywhere, the ecosystem of it, that’s really what I really love about it.

---

Continue to part three of the series.