The Evolving Normal: Resiliency and Lessons Learned from a Time of Disruption

In times of change, uncertainty, and disruption, the ability to adapt and innovate is critical to success. The shock and duration of the pandemic, along with worsening climate consequences have underscored the importance of resiliency and preparedness.  

SF Business Times Publisher Mary Huss recently moderated a Comcast panel discussion reflecting on the last 20 months, and what businesses should be thinking about in the coming years. Guests included Kristee Cominiello, VP of Comcast Business for California, Jeff Votaw, Comcast VP of Engineering, and Tony Hildesheim, Chief Operating Officer for Redwood Credit Union. 

MARY HUSS: How were Redwood Credit Union’s operations impacted over the past two years, and what does work look like today compared to a year or two ago?  

TONY HILDESHEIM: We’ve all been impacted by the rising number of fires over the last several years, and the subsequent power outages. Then came the end of 2019 and we ended up in the middle of a pandemic… while still having fires.  

Resiliency isn’t just about systems. Yes, you make sure your systems can survive a fire with backups and disaster recovery plans, and you have a network configured, as we have worked with Comcast to reinforce our network so we can maintain connectivity. But resiliency is really about how people endure these disasters. During the pandemic, all our back-office employees moved home. We had to keep them motivated and focused on our mission while increasing our bandwidth through Comcast to keep them connected.  

Then in the middle of all that, the fire impacted our ability to serve our members and keep our staff focused. We really worried, frankly, about how our team would work through a pandemic, in the middle of fire season. Thankfully it became a non-event in a lot of ways because we had enough resiliency built into our network. Most of our team already happened to be on a Comcast connection, which stayed up during the crisis. Because they didn’t drop their network connectivity, we were to serve members.   

MARY HUSS: Are there things you had to invest in for that technology infrastructure to work?  

TONY HILDESHEIM: We had already made Comcast our primary network, running out to our 25-plus branches, and to the internet. We increased our bandwidth with one phone call and within minutes we saw the difference as staff dialed in from home through our VPN. We increased our VPN licenses, and we were able to support roughly 300 people working from home.  

MARY HUSS: I’m curious what Comcast is doing to support its customers who may be struggling to adapt and stay resilient.  

KRISTEE COMINIELLO: While we certainly support connectivity and provide the network to help support businesses, at the end of the day, it’s about people.  

A few things we’ve been doing: We’ve got multiple services designed to meet any customer’s needs, whether it’s working from home or supporting an entire network. Certainly, we put an emphasis on security. As the pandemic has gone on, we’ve seen more and more need for security in addition to bandwidth. The other thing we’re able to offer is support for businesses that may not be able to manage their IT needs on their own. We can take a load off a business to help support them, not just through the network, but through services they’re unable to provide themselves.  

MARY HUSS: What do you see as some of the biggest threats to business continuity, especially in terms of broadband networking? How are these threats growing and what are you doing to address them?  

KRISTEE COMINIELLO: I think the biggest threat, as everybody probably knew, is not being proactive in evaluating your needs. And that’s what Comcast is here for.  

When we started experiencing this pandemic, we had teams proactively reach out to our customers to assess their network and provide bandwidth upgrades. Many customers realized they had holes in their security, and we were able to close those gaps as well.  

I think the best thing any customer can do from a long-term standpoint is constantly evaluate an ever-changing environment. While things are returning to the “new normal,” the reality is, particularly over the next 6 to 12 months, companies are still going to be figuring out how they want to work.  

JEFF VOTAW: From an engineering perspective, we’ve been innovating faster speeds, more capacity, and again, greater reliability. We do this in a number of different ways including redundancy, diversity in our network, and self-healing.   

Our network today is so much smarter than even a year ago, and it self-heals. We have artificial intelligence to not only identify issues, but to autocorrect these issues and find alternate ways of delivering those signals, packets, or services without human intervention. We’re also able to identify where something may be beginning to fail so we can address it before the customer ever experiences an issue.   

MARY HUSS: I’m curious about some key insights Comcast has gleaned from the past few years of disruption, and how these might be informing your next steps.   

KRISTEE COMINIELLO: In a 2020 IDC study, about 65% of organizations said they were happy with leadership, brand, and operational resiliency on the offset of the pandemic, but only 59% were happy with technology resiliency, showing there’s still work to do. 52% of organizations plan to improve the resiliency of their technology.   

MARY HUSS: How have your customers been asking for help to address business resiliency?  

KRISTEE COMINIELLO: Customers are using the cloud more. About 20% of organizations will use joint telco and cloud technology to support their needs. The other thing we’ve seen is awareness of some of the gaps they may have in their practices. More than 30% of businesses are going to prioritize connectivity to ensure business operations go uninterrupted. So, you have to prioritize that resiliency in your network like never before.  

MARY HUSS: Compared to a few years ago, before this sharp change, how have Comcast’s efforts to ensure network resiliency changed, and what’s stayed the same?  

JEFF VOTAW: What stayed the same is our mission. Our mission at Comcast is always to make sure our customers are connected all the time. We use the phrase, “Always On.” That’s been our core focus for the last couple of decades and continues to evolve into stronger and stronger technologies and innovations to ensure that happens.  

MARY HUSS: What are your business resiliency goals for this year and beyond? What have you learned from the past that’s informing your future?  

TONY HILDESHEIM: I think the biggest change is we’re looking at risk, redundancy, and recoverability from a much broader perspective, down to the resiliency and security of individual employees. How reliably can they show up either on video or at home, or come into the office during a disaster? Because that’s when you most need your top talent available, to make sure they and their family and their household is taken care of so they can work. Otherwise, your business is not going to survive  

MARY HUSS: How can businesses focus on long-term growth while they’re simultaneously addressing current threats?  

KRISTEE COMINIELLO: Go back to thinking proactively about your network, not reactively, to being intentional about listing not only your needs today, but what you anticipate they will be six or 12 months from now. Choose a network that enables you to be proactive and offers the flexibility of both short- and long-term planning. Security is one of the places in which we’re investing the most. We’ve got an active core platform, a software-defined network that allows people to have security technology across small and medium businesses. Our managed router and firewall are where the IT specialists come in and help.  

We’re really doubling down on unified threat management, as it allows our customers to integrate from a cloud-based perspective with that SD-WAN. Finally, we have our security edge product, which safeguards from cyber threats, web content, and search results, and automatically scans for new threats every 10 minutes. All these products work together and enable you to build the network that lets you be agile, flexible, and resilient.  

MARY HUSS: What’s on the horizon for the Comcast network and engineering team?  

JEFF VOTAW: We are always going to push the boundaries of what’s possible, which is the exciting part of our industry. We support a multitude of business customers, residential customers, residential customers working from home, and big businesses. In big business, milliseconds matter, so we are continuing to grow our fiber network, reduce the routing, and slash the time it takes to get from point A to point B.  

For the rest of our customers, we’re on a voyage together. The road to 10G is what we call it internally. For us, it’s about providing all our customers with a minimum of 10 gig symmetrical speeds. Obviously, we can do a lot more than that for business. but today 10G at home is a dream. We’re going to make it a reality over the next several years  

MARY HUSS: Fast forward to March 2024: How will businesses and workplaces and technology have changed? Are there some specific things you might point to that might be part of the norm?  

TONY HILDESHEIM: We see several things happening. I don’t think it’s any surprise that you’ll continue to see hybrid work environments. I don’t see my company at least going 100% remote just because the cultural impact is so great. Also, we get about three million logins a month to our online and mobile banking platform now. What’s fascinating is it’s shifting so a little over 60% of those are exclusively mobile logins. Mobile users are interacting more and more with our business, so we are investigating tools to let our staff work on mobile devices as well.  

Even more in the future will be virtual reality and augmented reality, where you’re starting to see more businesses have a presence in the virtual space. I believe we’re going to see a shift to some activities being done in virtual reality. This means more data use, which means more bandwidth, which requires more resiliency, which creates more availability for your team.  

KRISTEE COMINIELLO: More than ever, because so many interactions are happening digitally, individuals want to be able to interact with businesses on their own time, in the way they want to, and want it more personalized.  

For a transcript of the full conversation, visit the San Francisco Business Times.