Application Security

Building security training for developers in 2024: Is it really worth it and how to proceed?

Alexandra Charikova

Mar 5, 2024 • 9 min read

If you're a security engineer, the chances that you've struggled with your development team to implement secure coding practices are high. If you don't have the right culture in your organization, it might be even harder. Developers consider you as an adversary, rather than an ally that's here to help them. It feels like an uphill battle day in, day out.

We had great training at Segment, where we had two parts where every single new engineer had to go through it. We had to think like an attacker. And what it was is: we give you the tools that attackers have and you start exploiting a vulnerable application and that really helped people sort of figure out. We show them how they connect, exploit it, and then we'll give them challenges and say - can you find this type of vulnerability in the system? So it really made them think about how people are gonna be attacking their software and hopefully helping them think about how they can prevent that. And then the second part of that training was security code review. So how do you look for vulnerabilities in your code or other people's code when you do a review? And when you have that first collaborative exercise with developers, when they join the organization, they don't think about you as an adversary. They think of you more as a peer, and it really helped shift the culture within the organization, - Jeevan Singh, Director of Security Engineering at Rippling on the Elephant in AppSec Podcast

But what can you do? Security measures need to be integrated early in the development lifecycle, especially since many practitioners advocated for shift left approach within last years. We've got you covered - we've curated insights from industry experts featured on the Elephant in AppSec podcast and are here to guide you.

In this article, we'll explore the motivation behind training for developers, how to tailor your programs for success, embrace diversity in participation, the role of security champions in sustaining engagement, and how to maximize return on investment.

Curious? Dive right in!

The motivation behind building security training for developers

The mindset of developers now is... You're not touching my computer. I don't want you to slow my build. I don't want any more tools on here. You're not monitoring me. You're not this. You're not that. That has to change. - Mel Reyes, CISO & CIO on the Elephant in AppSec podcast

You might be asking yourself, why should you even care about investing your time and energy in training your development team, especially if there is resistance?

The motivation behind investing in security training for developers stems from the ever-pressing need to protect your organization and mitigate cyber risks. The need for speed might overpower security considerations - because developers need to ship, ship, ship. By equipping developers with the knowledge and skills to identify and address security vulnerabilities early in the development lifecycle when they actually ship their code, you can protect your applications against potential breaches right from the start.

Your goal should be to change organizational culture and it starts with teaching developers:

We started teaching developers. We taught them in a way that was very collaborative and engaging. And it completely shifted the culture there. One of the things that I was completely surprised about it. But everyone really started caring even more about security, - Jeevan Singh, Director of Security Engineering at Rippling

You can't go around it.

Tailoring security training programs for success

Ready to take the first steps? By talking with experts, here are the foundational pillars for successful training programs:

Collaboration

Collaboration is essential for successful security training programs. Engaging in open dialogue and partnership between security professionals and engineers can lead to better outcomes. Engineers often possess valuable insights into their systems that security experts may lack, making collaboration crucial for developing effective training strategies.

Definitely on the security strategy side of things, I feel so some of the things that engineers are much smarter about their systems than security people are. And mostly because they're building these systems and we get maybe an hour or two with the system and we provide the guidance that we can provide. But like, if you have that partnership and discussion, you'll get to a much, much better place, - Jeevan Singh

Engagement

Engagement is key to ensuring that security training programs resonate with participants and drive meaningful change. Training sessions should be interactive, relevant, and tailored to the needs and interests of the audience.

Engaging participants through hands-on activities, real-world examples, and interactive discussions can foster a deeper understanding of security concepts and encourage active participation.

Because the engineers have already decided this is useful to them and they've proven engagement, it's a bit easier than it would be if we were coming in cold, - Laura Bell Main

By prioritizing engagement, you can maximize the impact of their security training efforts and cultivate a culture of security awareness.

Customization

One size does not fit all when it comes to security training programs for developers. Successful initiatives are those tailored to the organization's specific needs and challenges. That's why another key aspect of effective security training is its customization - to also suit the needs of diverse roles within a development team.

But, before you start customizing though, you need to ensure you have a proper baseline:

Baseline things, you have to have that baseline and that has to be ingrained in the culture, right? It can't just be, oh, you know, the people that go into the office need to know this. Once you get that baseline, then it becomes very specific and targeted on roles, - Mel Reyes

Then, when the baseline is up and running, you can consider Laura Bell Main's SafeStack approach, which involves breaking down training into manageable chunks, aligning with different stages of the software development lifecycle, and providing practical, role-specific content.

Empower volunteers as security champions

In addition to security training for all, you might want to implement a security champion program in your organization. This can further enhance the effectiveness of security training initiatives. The Security Champion Program Success Guide is the right place to start.

"What motivated me to write the Security Champion Program Success Guide was really from spending a lot of time helping to set up security champion programs within my companies. A lot of mistakes were made along the way, and the learnings from those mistakes led me to share more publicly to help others avoid those pitfalls.", - Dustin Lehr, CISO at Fivertran and the author of The Security Champion Program Success Guide.

Central to the success of security champion programs is a clear understanding of the role of security champions within the organization. Security champions are individuals who are interested in cybersecurity, want to learn more, and can become advocates for security within their teams. They bridge the gap between security professionals and their own teams, tailoring security messages to suit the specific needs and norms of their teams.

However, you need to empower volunteers, you can't force people to advocate for security. Reflecting on his experiences, Dustin Lehr, noted, "One learning that sticks out is the distinction between volunteer and voluntold participation. Allowing individuals who are genuinely interested to volunteer for the program leads to higher engagement and participation compared to mandating participation."

Dustin also underscored the importance of tailoring security champion programs to suit the unique culture and dynamics of each organization.

Sustaining engagement and motivation

While security training and security champion programs may yield initial enthusiasm and engagement, sustaining long-term engagement poses a challenge. All of the industry experts emphasized the importance of continuous engagement and motivation.

Process for engaging experience from the Security Champion Program Success Guide

How can you do it? Here are some tips:

You can leverage techniques such as gamification and recognition to incentivize participation. One of the examples is implementing a belt system to reward participation and contribution. You might be asking yourself, so what happens when someone earned their black belt? Why would they continue really to do anything? Here's some advice from Dustin:

"And at that point, I actually switched the motivation a little bit and I say, okay, now that you're black belt, help us tweak and modify the program itself. You get invited to the steering committee. Your ideas are now accepted and applied to the program in general. So that's switched the motivation from curiosity to development and accomplishment to now contributing almost power, right? Or applying creative freedom to the program itself and that really keeps people engaged."
You can break training into manageable batches and provide practical resources. Think Lego. Didn't you like to play with it when you were a child? (Or you may still do it as an adult; no judgment here!) Laure Bell Main SafeStack uses this approach: Everyone will get a path with goals and deadlines that deliver a little bit of training and some practical elements to them regularly.

These best practices should help you build a culture of continuous learning, sustain engagement, and drive long-term security improvements.

Overcoming challenges and maximizing ROI

Despite the potential benefits of security training programs, organizations must navigate various challenges to maximize their return on investment.

You need to

Understand organizational culture
Align program goals with business objectives
Dedicate adequate resources to support program initiatives

But how do you justify the ROI? Here's some advice from Laura Bell Main:

For a software team, a security incident can derail their work for up to 40 hours. That's an entire week where productivity takes a hit due to focusing on fixing, identifying, or researching the bug instead of driving product innovation. Our belief is that involving everyone on the team – developers, testers, analysts, architects – in addressing security can significantly reduce the number of bugs encountered, thus minimizing the time spent on remediation.

We're aware that it typically takes about three weeks for a team to rectify and test a security bug discovered in production. By enhancing security visibility and reducing the incidence of bugs, we can avoid these prolonged periods spent fixing issues.

Additionally, there's a crucial aspect of software development centered around minimizing toil – the laborious, seemingly pointless tasks that don't directly contribute to product improvement. By integrating security tasks into various roles incrementally, we can eliminate the need for cumbersome, end-of-process security measures and replace them with manageable tasks.

This approach not only streamlines processes but also adds an element of enjoyment to the team's work, reducing the burden of toil.

So is it a magic box that's gonna reduce all of your vulnerabilities? Absolutely not. However, by distributing security responsibilities across the entire software team, even in small increments, we can accelerate development, enhance marketability to larger clients, and minimize distractions caused by bug fixes. Ultimately, this leads to happier development teams and more secure products.

Embracing diversity in participation

One notable insight from all our conversations is the importance of inclusivity in security training.

"I've seen people from marketing, finance, and even customer service or sales teams express interest in cybersecurity and get to "black belt". They could come out from anywhere. It just shows the varied interests of individuals across different roles within an organization. You know, you might find people who are interested in cybersecurity no matter what position.", - Dustin Lehr

You shouldn't stop just at the developers. If you have people from other departments who are passionate about security, embrace their interest and empower them to lead by example:

The QA team were normally very shy, very quiet people, who just sort of did their job. They had not only signed up and done their training, all on their own, but they then organized a presentation and made all of the developers and the senior leaders come and then told them why they were all doing security wrong and how they were going to help them fix it. And so they'd gone on this adventure from nothing to deciding to do some training, to working together to then plan how they were gonna make things better, and then engaging their whole senior team to make that happen. - Laura Bell Main

This serves as an example of the transformative power of internal advocacy and collective ownership of security initiatives.

Conclusion

We hope our article will help you implement an efficient security training program in your organization.

Building security training for developers requires careful planning, ongoing support, and a commitment to fostering a culture of security awareness. By empowering developers to take an active role in cybersecurity, organizations can enhance their overall security posture and mitigate risks effectively.

💡 Want to learn more?

Listen to the relevant podcast episodes:

Want to read something else? Here you go: