Bill Marantz of Linode on automation, mentorship, and problem solving

Avi Freedman: Hi, and welcome to Network AF. This week I'm talking to Bill Marantz at Linode. We're talking about how he got into technology. We're talking about open source, NetBox, systems of truth and record, and how that ties to automation. We're also talking about mentorship, recruiting people when we're growing very fast and trying to grow the pool, and also about problem solving and how important that is, and how it can be really helpful to bring to networking if you're interested, even without some of the deep networking technical details background. So thanks, please join us. Hi, and welcome to Network AF. This week, I'm with my friend Bill Marantz at Linode. We'll be talking about networking, open source software, decision making, community. And let's get to it. Bill, could you give us a quick introduction?

Bill Marantz: Yes. I'm Bill Marantz. I'm the Director of Network Strategy and Interconnection at Linode. That's quite an interesting full of a title, but I wear a great many hats doing network engineering, architecture, down to DWDM and cable plans. And then also a lot on the sourcing and strategy and negotiation side. Working with data center providers, different CoLOS, working on ordering cross- connects, turning up peering transits in other capacity, and I'm also our peering coordinator.

Avi Freedman: Got it. Thank you. I have been a Linode customer for quite some time. I enjoy using it. And it's also cool to talk with other folks that are doing networking in Philadelphia, as this is my background. So how did you get into tech? How did you get into computers, and how did you find networking?

Bill Marantz: Well, I guess when I started it, that's what we called it. We called it computers. I always knew when I grew up I wanted to be in computers. I got my first computer in probably fourth grade. And then by the time I got to high school, they started actually having BBSs and dial up and some tiny LANs. And I said,"Well, this is really cool." I guess, in freshman year of high school. And then I went to actually a specialized technology school at the time, a Magnet school, starting from my sophomore year called High Technology High School. So I said," All right." I really wanted to go into this area. And when I started to learn programming, I was like," This is cool, but it's not something that I could do every day." And I said," I don't want to be an IT generalist either." And then I started learning more about networking. I said," This is cool." And the internet was really starting to explode at that time. And yes, you had the AOL, CDs and you've got mail and everything. But it was also... You started to see commercial ISPs. When I went off the college, we really had networks in the dorms. I said," Okay. This is cool. This is what I want to do, work on the fundamental infrastructure that gets users to whatever applications are out there." And at that time it was mostly email and some basic business websites. And then it quickly developed. And then while I was in college, peer to peer file sharing really started mostly for bootleg uses. But at that time, it exploded. And I was done with college, I said,"Okay, I want to go work for an ISP." I had worked for AT& T actually.

Avi Freedman: Oh wow.

Bill Marantz: My high school had a really cool mentorship program. It was science and engineering technology. But it was really more math, physics, chemistry, electrical engineering. And they weren't really sure what to do with me. They said," Oh! We have someone AT& T." I said," Okay, I'll go look at telecom. That sounds really cool." And they kind of put me in a basement with a box and gave me a Cheswick and Bellovin's book on firewalls and said," Why don't you figure out internet firewalls?" Which was really in its very infancy those days, and said," Here's some books. Understand the TCP/ IP. Understand routing and give a presentation at the end of it." I said," Okay. This is really awesome." And I figured it out and got it working. I said," Well, this is definitely what I want to do for my career." And then every kind of summer and winter during college, I went back to AT& T Bell Labs and helped them out, and did a lot of general IT support and some Windows server support. But there were also a lot of Cisco routers and other stuff. And I just started playing with it and said," All right. This is really what I want to do."

Avi Freedman: Cool. It's great that you had that opportunity. When I showed up at college, I was interested in distributed processing. Obviously, it's related. You can't really distribute anything if you don't have things connecting. But I fell into the networking side of it and got into optimizing multi- user dungeon games. So learned Unix IPC, but also we did on VMS a little bit. But not to my taste. But also then sockets, and decentralizing, and streams. We even did some OSI stuff, which I think we only now have in ISIS. I think that's the only remnants of it that are in networking. And that was back in the days where a lot of stuff was open source. But that didn't mean you could get it to compile between ESICs, IRX, AT& T, BSD, Otrix, SGI, HPUX, whatever. Some of the libraries with this. But you mentioned open source is something that you use and is a passion. Curious nowadays what your interaction with open source is, personally in the job, networking.

Bill Marantz: It's mostly around Python scripts that can do networking things. You manipulate IP addresses and the like. Or render, given some set of data and a templating language, render that template or inaudible that template and say," Okay, what changes occurred here?" Most recently, I've been using NetBox a lot. And I find as everyone's building out and scaling these awesome cloud networks, globally, there's a lot of... Often a lack of appreciation that there's still something physical there. And NetBox is great for," Okay, I want to trace a cable from point A to point Z. And I have a DWDM network. And I have a cross connect in a carrier hotel that's bringing back, N by 10 gig or N by 100 gig circuits and the knock needs to be able to troubleshoot that. And how does that all connect? And how does it multiplex up in. Okay, I need to swap out an optic. I need to change a cable. Where do I even look?" And NetBox was great for that. And then once I have all that information in there, they have an API, so I can and query that API and use their functions and automatically generate network interface so then say," Okay, this is point A, this is point Z. This is the carrier at point A, and this is the dark fiber that it rides to a carrier hotel." So someone can jump on the router and say," Okay, what's going on here now? Okay. Everything looks good logically. Where is the physical tie in? And how does it tie in? And just immediately look that up in NetBox and using... I don't program our alerting systems, but we do use a lot of great open source, Prometheus and other things. And they support labels, and you can put that label and it imports it from the interface description directly into that system. And then when it fires an alert and says," Okay, this is that utilization, or it's hit a certain threshold of errors, light levels, what have you." And there he says," Here's your NetBox." If you look on that, you click it a button, and it's going to show you that the path from A to Z, potentially across dozens of data centers and dark fiber and cross connects and patch panels. So you can really start troubleshooting.

Avi Freedman: Yeah, it's interesting because when we started Kentik, we built in the ability to be very flexible with that tagging, as you mentioned some of the time series and other open source systems have as well. But one of our challenges has been getting... people sort of giggle hysterically, or sigh, or cry, or fall under the table when we say," Where's your source of truth that we can pull from rack locations." Or what an application is to IP address mappings, or whether it's CMDB, IPAM, stuff like that. And it's been great to see the growth of NetBox and NS1 on the DNS side has really, I think, taken on the project and invested heavily and is focused on that. So it's great to see. It's really hard to automate when you don't know what the parts of the plane are, or in the Kentik sense when you have no dashboard. It's really hard to make a flight plan and make things go. Is Python and network libraries... Templating, is that something used across the group inside Linode?

Bill Marantz: Yes. It's very widely used. I mean, everyone kind of... We don't necessarily contribute as a network team to open source, but we're using it. Everybody writes changes for templates and/ or the underlying data. But I mean, it really starts with that single source of truths and the data model, and then you feed that into the templates and then you get... The simplest thing, even though it's very complicated is interface descriptions. And then that can feed all of your monitoring and alerting, as you said, just tie right into inaudible and classify your interfaces as peering transit backbone, customer data center interconnects, or whatever else you have to then capacity manage, or cost manage and send up alerts to the right people.

Avi Freedman: Yeah. We definitely see that the older the network is vintage- wise, the more likely there is to be at least some part which is the dungeon of if there is an interface description, it's wrong, and most usually there isn't. But less in web companies and more in companies that evolved since Sneakernet was a thing, where you're just starting to connect stuff together. But that is really what we see as the basic level of... If you're just doing that life cycle, keeping things clean, starting what you're describing with Python, that's really 90- something percentile actually for companies. There's all this vendor juju about," Oh well, everything is automated and everyone's behind it. It isn't talking to their computers, and like Scotty in the Star Trek, whatever movie that was." But, that's the struggle that everyone has right now, I think, so.

Bill Marantz: Well, there's a bunch of fear of missing out, and they want to sell their software. And sometimes their software implies a design that may not solve your problems.

Avi Freedman: Again, that's another topic for maybe a panel on automation. But I was surprised how much you actually have to know how the router thinks about itself in the CLI to use some APIs. So it's not like you wave your magic wand and say," I just want to do this to that." And you have a policy affected. And then a lot of the most advanced intense stuff is for very, I'll just say, constrained solutions and topologies, not for internet facing stuff that we actually face. We get asked to solve that all the time. That is a hard problem. That is-

Bill Marantz: A lot of those tools are, I don't want to discount them because a lot of them are great, but they're really for data center causes. And in fabrics, we have a very fixed topology. I mean, my realm of background is backbone engineering, and no two routers fortunately are alike. So it's more, instead of saying," Okay, this is what a router looks like." It's like," No, these are all the different components that could be. There's peering, there's transit, there's backbone links, there's X routing policies. And then you combine them to make a router." Instead of strictly defining a router, you define all the components that go into that device.

Avi Freedman: Yeah. I think we'll see a lot of progress on that, and I think we'll see even just... We've had RANCID forever to do config monitoring. But sort of the inverse that make it so command the NCCM network config and changing config management, doing that at scale, that's still something that's evolving, which again, we get asked to solve. But we sort of view that as outside the purview for our focuses right now of observability. So yeah.

Bill Marantz: But one more thing I'd like to... you do the network configuration, you do the templates, you have the templated interface descriptions, you have what you think the topology is. I've always wanted to write something, I've never had the time, to reverse engineer that with LLDP and say,"Well, this is what I describe and I think exists, actually exists," because LLDP is really good at telling you what's actually there.

Avi Freedman: Let's talk offline about that. We actually have a script that we've done. In fact, I didn't tell you about that commercially. But it's like when you have, as you said, Python APIs, sometimes you can put stuff together. And so when we see people asking for features that aren't exactly there, but you can use this platform, you can use NetBox to grab some of that. In fact, on some devices, even if you don't have a CLI scraping mechanism, you see LDB in the inaudible or in streaming telemetry, so then you can combine these different things to put it together. Doesn't always have to be the most elegant thing. In fact, I call it exigent engineering, everyone else calls it Avi pro code, but it's like, let's just demonstrate, let's build a running specification, this could be done, and then have the smart people that actually write software professionally look at how to do that better. So we can talk about that use case. Cool. So I am jealous. I had a wizard who had a lot of bell labs experience who got me into Unix when I was in my teens, but I did not get to work at the Mecca bell labs. My uncle shut me off from the ARPANET after about 30 minutes of me poking around actually pre Unix, even like PDP days. So that was a mentorship program that you were on as part of the high school? Was it...

Bill Marantz: Yeah, it was a senior it senior mentorship program. And then I extended it more because I was so I was so into it and you couldn't go to a community college. They didn't have Cisco academies or any of that at the time, there were a few vendors. What vendors existed didn't have formal training when I was looking for... To go onto university for a four year degree. It was it or information systems. It was not... There was no network engineering that wasn't the field. So I had to find my own way there. And I definitely learned a lot more during my summers and winters at inaudible than I did in any of the coursework. Obviously I learned more in the coursework, but not necessarily about networking. And then... I guess this was my senior year. I did end up taking a graduate level class in specifically computer networking. It was taught by, it was an adjunct professor, but he was the chief network architect for the city of Pittsburgh. This is really cool. But at that point it was a night class. It was mostly for people getting an advanced or a second degree. And it was, unfortunately... It was a very good class. It just wasn't... I'd already a achieved most of those skills by that point. So it was like,"Oh, I wasn't earning all that much." But it was good to validate what I'd learned in different pieces on my own, from somebody who had really been doing this for years and said," Oh, okay."

Avi Freedman: So, there's the formal, here's how it works and distance vector and all that. But then there's the, just because you can do X doesn't mean you should, right? Just because you can inaudible doesn't mean that you should go to that as your first thing to do. How we learn these things.

Bill Marantz: Yeah. That today would be... I think it would be a very different course at a grad school today. Because the stuff that was taught then, most people that have come up through IT will have already known. And that would've been a lot more useful. It's like," Okay, the books and the diagrams and a lot of the sample labs, no one would do that in the real world. So here's the problem. And these are possible ways to solve it." Which is really the best way. And the best way is not always the same for every company. And they may have different technical parameters. They have different business parameters, they have totally different cost structures and financing structures or a much more established company versus," Okay, I need to get this do can do this." And prove it to someone that did not invest in this startup so we can really scale it."

Avi Freedman: Its really amazing. Even the non- technical side. For me, I remember emailing the may east mailing list, like" How does this BGP thing work? Do I just show up and get peering?" And I got an email from Bob Gibson who ran Case and was like" Call me." he was like," Well this is a grand secret. You can't tell people." But I'm like," What's an MRI and what's this and how does peering work or what's..." And it was like... I was lucky I had a mentor, I had Dan Ellis, Alec Peterson, I had different people that helped teach me these things. Because, again, in the nineties, the information to lay it out there for people that weren't already deep into networking, I'll say, I call it for curious inaudible admins, which is where I was coming from at the of time was not so great. So sounds like that was a great experience at AT and T. How do you think about mentorship today? As you're running things, leading building teams?

Bill Marantz: Yeah. I'd say in the early part of my career, I was so busy. I didn't have extra cycles. And then when I was at Salesforce, they said," Okay, to move up in the organization, you're going to need to become a mentor and start creating future leaders." I said," Okay, great. If you giving me the time to do it, I would love to do it." So, being in the engineering organization was a more operations focused organization. I'd sometimes watch the tickets or be on the email threads, interject myself and say," Hey, maybe we could do it this way." And then some people were very open to it and some people were not, and the people who were open to it then came, started asking me more questions." Okay. Well, why did you do it that way? What are, what are the other options? Thank you for that. I really like that. That's awesome. I want to learn more." Or people... I wrote a lot of documentation at Salesforce and people would just find that documentation and say" This was interesting. I'd like to learn more. If you're working on a projects, Bill, can I be an understudy? Or could you dole out part of that responsibility?" I was just happy to do it. It was fulfilling. I think it really grew both sides and you learn a lot more and you can really have a crisper delivery of things. If you have to teach it to somebody else. Teaching is harder than doing. It's like, you do it all the time. You forget. But now," How am I going to explain all those steps and why I do that? How do I make that simple to someone else who may not have the years of experience as a technical background, or even know some of the underlying technologies." So how can you really simplify that and draw diagrams or simple words and get people moving. And I find most people appreciate that and want to learn. I developed some really great long term friendships with my mentees and I still keep in touch with, and they occasionally ask me technical questions. We mostly talk about personal stuff now, but a technical question comes up here or there, and I get my input. I say," Hey, I looked at it this way" They said," I didn't, that's great." I like different viewpoints. And there's so many different ways to solve problems in networking and the right solution for company A could be very different from the right solution for company B. Or they could be the same, just not necessarily at the same time, due to organizational maturity or skill levels. It's just definitely been" Okay, I didn't do this because we didn't have the monitoring or automation to build that kind of network at the time, we didn't have the skill set to support it." But that doesn't mean that's not aspirational. It's like," Okay, we're going to get there. We're just not there yet." And just break up that project into the smaller chunks and assign owners and say," Okay, we're not there. Or vendors could not be there." Sometimes our needs are ahead of the technology. I know in some of your other podcasts you were talking about SONET. inaudible over SONET. So," Oh man an ATM." I was like," Man, this be so much simpler." And then carrier ethernet became popular and ethernet... crosstalk

Avi Freedman: All the people that wanted NPLS dealt with bugs while they were getting it. There's always trade- offs. And I can say that with the hindsight of decades, I don't feel that old, but when you teach people things and are helping people, many of those people will go on to... I don't want to use a loaded word, but surpass. At any given time, there may be something that you need to learn or an introduction that you need or something they come around in business. So there's pluses to it also. I guess the trick and something I could do better too is, how do we show people that it's encouraged? I think maybe sometimes we expect people to model the behavior, or I expect people to see that when people engage and ask questions and have the bright, shiny eyes that good things happen. But maybe there's people from different backgrounds who are shy, or who don't look for non- structured interactions. So it sounds like Salesforce had a good structure for that. And it's something we think about. And other companies think about, is how do you, how do you encourage that mentorship and connecting people? So.

Bill Marantz: And some people are assertive and some people are not, but doesn't mean," Okay, I've seen this person do this good work. And they haven't reached out and asked for more." But I say," Well, I have a project and they've done something similar or shown an aptitude, at least give them a chance and say,'Hey, would you like to work on this with me?'" And, worst they could say is no and the same in the other direction. Somebody could always come out and say," Hey, Bill do you have the time, or do you want to work with me?" Worst I could say is no. Usually I'm going to say yes. I can't say yes to everybody at once. Right?

Avi Freedman: Yeah. No, that makes sense. We definitely think the same way at the same time as we're growing, try to... we're trying to think a lot about that right now, how do we formalize that? And so, that's something we're spending time on. I've seen other companies of similar size a few hundred people where you're not quite at the size where you have process and everything formalized, but we want to take the best of what's going on and try to scale it. So.

Bill Marantz: Yeah, I don't know if the actual process being formalized is super helpful, but I think the baby step of at least providing the matching is like 90% of the solution. It's just finding people that are willing to teach and people that want to learn and mixing... Putting them together and maybe their styles don't work and maybe they need to move on to a different pairing, but...

Avi Freedman: Well, and as you mentioned at Salesforce, also having the time to do it, right? The acknowledgement that it does take time, but then good things happen at the end. Or maybe people decide that isn't for them. And they find another path. So.

Bill Marantz: Well time and patience, because there's always the quick way. Is to give somebody the answer, but they're not going to learn all that much. It's like," Okay, well, how would you do this?" Or," Okay, this is wrong. Can you think about why it's wrong? Or have you thought about what would happen if this happened?" And it's like," Oh my, this is not a redundant design or, well, I didn't anticipate that, or I'm draining a router and I'm doing all the right things. I'm doing them in the wrong order. inaudible I've turned down ISIS, and now uh- oh what happened?"

Avi Freedman: Sometimes you have people that are like," No, you're wrong. I will show you." And what's fun is usually 80% of the time they'll like," Oh, okay, you're right." 20% of the time you might be wrong in your assumption, or there might be a different approach. And then that's also either way is a good outcome, but you're right. You have to allow crosstalk

Bill Marantz: I recall a very long discussion about that in one scenario. And they're just like," No, you're wrong." I said," Let me diagram out exactly our topology." And why I'm right. And," No, no, it's always going to pick a consistent route." I'm like," Nope."

Avi Freedman: Well, people think that BGP does a lot of things that it doesn't.

Bill Marantz: Doesn't do.

Avi Freedman: And including really the deeper down you squint at best path selection. People think networks are much more deterministic than they are, and also underestimate bugs. Which is a whole other topic of how much you have to do planning. People are like," Well, you can't consider vendor bugs in your disaster scenario." I'm like," Oh, oh yes, you can. You should." Because they occur in certain ways. In the nineties, it was like, put the" inaudible card in and sparks shoot out the hissy port. But today it's more consistent kinds of things that are just what you shouldn't do. Or it's like," Yeah, I know it says you can have that many next tops, but not really. They don't really mean it. It meant that their automated test didn't break, but that's probably not a good idea." Again, it can be hard in the non- network world, too or the network world where we talked about open source. The line is blurring between what's a service mash and a network mash and all that. You use a new tool, if you don't know how it fails and you're not actively developing it, that's a dangerous position to be in. Because software is not perfect and fails all around. We're lucky that we have Linux underneath and that is run pretty well that has gotten very stable, but definitely things to look at, speaking of that...

Bill Marantz: Or have a controller that sits in your network, that's then part of the path that you may break. And now the controller can't talk to...

Avi Freedman: Oh, so many multi levels crosstalk

Bill Marantz: And now your network won't work.

Avi Freedman: So speaking of that. You mentioned this before, problem solving, right? Which has always reminded me of my physics teachers who were like" Starting from the basic laws of physics, derive this and this and this." And it's like, because networking is complex or lots of simple things that work together complex in especially internet connected, real world applications. How do you think about problem solving? Selecting, hiring, training mentoring around problem solving and networking?

Bill Marantz: I guess from my perspective, it's really crisply defining the problem in the user space and understanding how someone's going to use that application or that part of the infrastructure, or how many things are layered on top of that infrastructure. So then what features you need to deliver and do you need one infrastructure? Do you need multiple infrastructures? Do you cater to the least common denominator there? Or how do you really approach that? And then just looking at a bunch of diverse backgrounds and saying," Well, how did you solve that problem? Or how would you solve that problem if you never solved it?" And just coming at it like," Okay, I've never solved this problem. I don't necessarily even understand how the underlying protocols work, but if I was going to solve that problem, or if it was a Greenfield, what things would I consider?" And I just find that's very helpful. That there's somebody who's to always solved the problem and always solved the problem one way could be a lot less useful than someone who solved the problem five times in five different ways or someone who's never even solved the problem, but just is curious and wants to really ask and understand and say," Okay, if this broke..." Or," Okay, well now we're not using this device in a car we're using this device underground." Right? You know. How does that change the problem space?

Avi Freedman: Yeah, it's interesting.

Bill Marantz: I know that's an extreme example, but.

Avi Freedman: A lot of companies... At Kentik, we don't do the crazy either stand up, put a whiteboard and reverse a binary tree or the," How many light bulbs are there in the United States?" But I think it's definitely true that even if you don't get to the right answer, understanding how people can break something down and attack it, whether it's learning or whether it's problem solving is helpful. It's another thing that, there's a lot of wisdom about that people might have might not have. And so we also think about how do we set expectations about, say an interview. What are the conversations we're going to have? What are you going to... Don't worry if you don't know the exact answer we're trying to get at this... Again, as we scale, we think a lot about that. But I hear you talking about, you can have all the network experience in the world, or maybe all the network experience in a manufacturing plant in the world and be the best at that. But if you don't think generally enough about problem solving and how that applies to maybe not even the technical, maybe even the business and crosstalk

Bill Marantz: Yeah, I've had interviews where they only... the questions they ask. It's so specific to their environment, and I did not work in that kind of environment. I hadn't really thought about that kind of environment. So I could give them a rough guess, but it would take me a lot more time than that interview to think in detail and say," Okay, these are all the inputs. These are all the variables. These are all the permutations. These are all the things that can go wrong." You always need to be think... To be a good engineer, you need to think about all the things that could go wrong and almost in that way, be a business person and prioritize them with probability, right? Risk management. Okay, is this likely? Is this not? Is this a corner case, or is this going to happen all the time? But when I'm interviewing someone, I really want to see their thought process, see how they would troubleshoot things. And I try to ask more open- ended short questions and just let them go. One of the things I really ask about is, Okay, packet comes in one poor on a layer two device, inaudible that other layer two device, what must happen? What could happen? What happens... Right? What happens in a layer two scenario? And then doing a layer three scenario. Okay, I'm going to go in one port outport, another route. Just to show... crosstalk

Avi Freedman: It's amazing how many people are fuzzy about... crosstalk

Bill Marantz: They're thinking about TTLs, they're thinking about buffering. They're thinking," Okay, this could be multicast. It could be GRE encapsulated. It could be VXLAN encapsulated. It could go into an IP sec PPN. It could be Ned." Those are all potential things that could happen on a layer three device. Right?

Avi Freedman: Well, and what you're looking for is relative to their background and claimed expertise, experience. You can always give hints, but it's not going to occur to a developer who hasn't looked at it that switches to Mac learning, versus they might have seen from layer three. But if someone's saying they've been running large data centers, then they should know that. Or draw the internet. Which again, let's say someone's a peering coordinator, but not a hands on, into... This still should have some idea at some level, and also gives you a good idea about, do people come at it from the application side, the infrastructure side et cetera. You talked about diversity of background and people that have pure networking experience could be very senior with very specific network experience, versus people that can bring an approach to it. The broader the set of people and backgrounds, the better questions you ask, the better answers you get. How do you think about getting a diverse set of people in a networking organization? Recruiting them, pulling them from other parts of the company? Or from the world where there's... especially at the senior level, were a distant vantage in some ways, just due to history and approachability.

Bill Marantz: Yeah. I guess part of... I wear so many hats, so I do interface with a lot of different parts of the company. So I do want to gauge interest and say," Hey, would you like to have a session? Do you want to learn more about that?" Because there's definitely more diversity when you look across certainly an organization like inaudible there's a lot of support people. There's a lot of developers, there's finance people, there's folks in those different roles that tend to be more diverse than just network engineers. Right? And I really think getting very young is very important, and this is something I've... it's been a challenge for me. And I've reached out and I just moved and now COVID is winding down. So I plan to reach out again. Is go into high school and elementary school career days. And everyone's just so tied into their devices today, it's almost neglected. It's like this magic just happens. I'm like," No, there's a lot of thought. There's a lot of engineering. There's a lot of resiliency. There's a lot of..." The scale to make watching whatever video you're watching or whatever social media site you happen to be hitting at that time, work at that scale is just my mind blowing. I think young folks would be really fascinated by that. So get them young. And I think a lot of people really want to be... And it's probably what happened to me. I knew I wanted to be in technology. I knew I wanted to be in computers, but I didn't necessarily want to program. I didn't want to necessarily design computer chips or choose signals and systems that detailed electrical engineering or optical engineering. And maybe... I don't think a lot of people know that is an option. They ask what I do, I say," Okay, I build the internet infrastructure."" What does that mean?" And when you talk to people and say," Okay, you're, you're sending data, you're making a phone call from Europe to the United States." And you talk about sub- sea landing stations and DWDM systems and other things, and cable ships and cable repairs, and people are fascinated." Well, how is it powered?" And you talk about that. People get pretty interested y that. So if we start... It's those people they already crosstalk.

Avi Freedman: Internet infrastructure is in the news today, right? It's been in the news, it keeps being in the news. It's relevant to us and how we work. That's a great idea, Aaron Kagawa, who works for Kentik talks about tech- not even specifically networking- in Hawaii, goes to different islands and talks to schools. I know Anna Clayborn was talking about inaudible was talking about getting math folks, which also is... earlier in careers generally more or diverse also, even in computer science. But getting math folks who study... Who think about parallel processing and algorithms and distributed systems, or... To in control systems. And to get them interested in the internet, from that perspective. I used to go to Temple university when I was still in Philly. Although to your point, I went to talk to seniors, which was much less useful than talking to freshmen would've been, we found some interesting folks that were interested, but they sort of... People had already chosen... crosstalk

Bill Marantz: What kind of messaging did those students run into earlier in their life that they thought maybe that wasn't a viable career path for them? But if we talk to them... To young... And technology's in just like every aspect of people's lives. And I talk to child A, and their thing is sports and child B, they want to watch videos. And inaudible is voice recording and in music, the technology's in there. You can apply that and say," Okay, well, I want to do video and coding at... In streaming, locally or globally at scale. And I want to produce this and I want to do a video or interactive hip- hop video, or what have you" Right? crosstalk

Avi Freedman: Perhaps now it's metaverse, right? With Facebook and their direction. Everyone, the kids are not aware of the edge computing hype, but if you really want to have ready player one, there's a lot of interesting network and software systems that have to be designed to really make that happen.

Bill Marantz: The other side is just like really investing in our schools and the technology they have. There's a lot of awesome, awesome teachers that may maybe with specialized in networking or an art or something else, and use this technology and distance learning. Now that Zoom is so popular, but it was... or other streaming technologies, after COVID. But it still seems like the classroom is traditional, that all the kids are still in the same town, tied to a teacher in that same town, or if you're really lucky it's done on a county basis. Well, if you find 50 kids across the nation that are really interested in this specific area of technology or, or culture or arts, and find an expert teacher and spread that person across all those kids using these awesome technologies that we now have to really help those kids. So focus on something that they're passionate about and not get lost in the system and not get through high school and say" I don't have an option. I don't have a choice." I'm either going to go to college or I'm going to go work in regional. It's almost like trades are forgotten. And we talk about mentorship and you've mentioned this before. It really it's really like a modern day apprenticeship. Right?

Avi Freedman: Right. Yeah. I mean, what's the VOTAC for this kind of thing? I think that NANOG is trying to trying an approach towards this. I think what you're describing, I could see working maybe if there was a core curriculum and a bunch of people that were interested in helping and maybe it wouldn't be the density in every class or in every grade or in every district, but finding people that were interested in effectively, a more interactive Khan academy approach, maybe targeted high school versus college level could be a good way of helping people get more educated. And of course, people want to know about cloud and they want to know about architecture also, but to be able to do that on networking would probably be helpful. And we have some of that for people that we want to train up at Kentik and at Akamai, I think the second week I did a class on BGP and we had half the company squeezing in the room to try to understand. They'd already achieved massive success, but actually peering and how negotiating and policies and all that was sort of vague because came from computer science and a practical background, not from a networking side. So.

Bill Marantz: Yeah, I've seen links really step into that space too.

Avi Freedman: I'll have to take a look at that.

Bill Marantz: Talk about," Oh, okay. How do we grow Ixs?" Well, people need to know that they exist. They need to under understand BGP. They need to understand that there's, it used to be a very cost savings and it still is, but there's just a lot more parts of diversity and capacity and being able to handle bursts. Traffic is all over the place with geopolitical events. And now so much of sporting events and other things are just live and in real time, and this network goes from zero to 900 in seconds. And then back down again.

Avi Freedman: inaudible always had elements of cost, performance control, and diversity, right? Having diverse paths gives you control. And definitely in the nineties, it was more about cost at$2, 000 inaudible megabit. And now transit can, can be cheaper. In fact, for many of our customers, transit is cheaper, but the reason they don't do it is because then they're dependent now there's trade offs. If you announce your paths to 300 networks, there's 300 people that could do something bad with your routes, but especially for content heavy folks, having more paths means you actually have tools to fix things. So for content, you can't move to a CDN or multiple CDNs. For the most part, not everywhere, but outside of South Africa and other places, the cost is not the primary factor there now. And, and... crosstalk

Bill Marantz: Path diversity's a major drive. There's really no downside to path diversity today and routers can hold the paths. There used to be a downside. Right? But we're not... The routers end up blowing up on Ram these days.

Avi Freedman: Yeah, no, it's working overall pretty well. So... crosstalk

Bill Marantz: I mean that it works and that we just talk about, get into the peering community and mail this and just like," Oh, well, I just sent an email to someone in 10 minutes or 10 days later they responded and we just set up a BGP session, and now we're exchanging more traffic over that link on a shared IAX and backbones exchanged in LA nineties." Right?

Avi Freedman: Right. No, I know it's pretty crazy. I remember a gentleman came up to me at a Nanog in 80... Sorry, 99, I guess when I was at above net. And he was like," You don't know me, but I'm Aaron inaudible, and I'm behind I'm probably your biggest source behind, inaudible unit." I was like," Oh!" As I think it was 5. 68, whatever bell candidate inaudible And he was like," Yes, how'd you know?" I was like," I watch this stuff too." Although it was much harder to figure that out back then before when net flow was barely at all, much less being able to take the data and use it. It's fun. But again, separate topic, just the philosophies of peering and again, the background, a lot of it is about economics and politics and not just the technology. So it is a beautiful thing when you can find people that are compatible and all of a sudden the customers get happier, that's ultimately the best outcome.

Bill Marantz: And you'll meet some people and they're like," Oh man, this company's so rough." Then you actually, they're not going to appear. You meet the person and it's not necessarily them. It is the business decision. They could be the most, the nicest, most helpful person and will introduce you to other peers and help you out. They may not be able to help you out... crosstalk

Avi Freedman: And, they move... crosstalk

Bill Marantz: ...for pairing, with big eyeball network a, but that doesn't mean they're not going to help you out and be valuable or learn something from each other. Right?

Avi Freedman: Yeah. And it's also, it is a direction I encourage people to go if they're not interested in as much in the technology side. In some sense, it can be more welcoming because it is a much more diverse community. Because people come from even more different backgrounds than the... Let's face it, in networking, you can have very spirited debates about architecture and what's best here and what's best there. Now we have the same debates in interconnection, but there are more likely to be people that may have a similar background to you or look like you, or in those environments, especially when you go international. So I think they've done a even better job of keeping that up, and getting people from a business side, from a technology side. From sometimes a marketing side, and then sometimes even getting into switch hats and get into more of the technology. Although that rarely comes into backbone and cloud networking, usually it stays on the internet interconnection side. So.

Bill Marantz: But as an industry, we just have so many job openings. We need to diversify, we need to focus on outreach. We need to get the people early and maybe get some of those electrical engineers or programmers before they go down that path and say," Well, networking's really cool." Or other people say they want to be a business hybrid. You go into peering. Right?

Avi Freedman: It's not just college. It's not just high school. It's not just internal transfers. I think we need a mixed, diverse approach to it. And we'll give it some further thought. Cloudflare's done a great job with their learning center. We're trying to do something similar. A lot of other companies are also about," How do we curate and help people do that for..." And not just networking, but infrastructure in general observability, system design. All these things are related and the lines are blurring over time. But it can be hard to get into if you're starting with an iPhone. And trying to understand everything underneath, that's a pretty big gap. It's a bigger gap than coming from an eight big computer and trying to serial connect it, and then understanding networking. So.

Bill Marantz: Yeah. I appreciate the outreach you're doing, and a lot of people are starting to add those sections on their websites, whether it's white papers or architectural diagram, they're just general learning resources. And that's awesome. Unfortunately a lot of them still are behind, not necessarily pay walls, but," Oh, well, you have to give me information so I can market it to you." And that's going to turn off students and as a marketing person they're not going to be chasing 10 college students and they're not going to be buying anything. They just want to learn.

Avi Freedman: Yeah. No, definitely something to think about. Well, thank you for the time. And for sharing, definitely seeing a lot of the pattern of folks using NetBox and trying to wrangle things together and understand how to do automation and hiring, as you said, is a challenge for everyone and need definitely some creative ways of thinking about it. If people want to find you Bill, how, how can they reach out to you?

Bill Marantz: Best to just find me as Bill Marantz on LinkedIn.

Avi Freedman: Okay.

Bill Marantz: Usually the best way to contact me.

Avi Freedman: Cool. I'm Avi Freedman on LinkedIn, Twitter, avi @ kentik. com. If you like this kind of format, feel free to find, like, subscribe to network AF. If you have ideas for other interesting topics, areas of discussion, please ping me as well. Thanks again, Bill.

Bill Marantz: All right. Thank you, Avi.