NVIDIA Architect Interview Questions
& Process

The whole process took about 2 months. I had 2 Teams interviews over that time, one with a project member and one with the project lead. They asked basic questions about my resume and some fundamental architecture questions. After that there was a panel round where I interviewed with four team members. The first panel interview went okay; during the technical question I had to do by hand I messed up a lot, but still ended up with the right answer. The second team member didn’t ask many "technical" questions, mostly stuff from my resume, but they asked me about things I didn’t actually work on. It felt weird that when I said I did x,y,z, they’d ask something tangential that, based on the rest of my interviews, they didn’t really seem to care about. I said something correct but with the wrong reasoning to that, but overall I answered the rest of the technical questions fine when they were related to my experience. The third interview was a complete mess. The interviewer never fully defined any of the problems. It was like being asked to do the operation "what’s 5 and 6?" over and over, and when I asked for clarification they’d just say "what’s 5 and 6 together?", and I had to keep guessing if they meant 11 or 30. I should’ve realized this earlier, when I referenced a common operation in PyTorch/NumPy/TensorFlow and they seemed clueless about the notation. The final interview was great but had the same issues as the others. They asked if I know "tensor parallelism" and my immediate thought was "what a dumb name, you basically have three tensors: input, output, and weight, and only one of them is parallel." I felt this was my strongest interview because at that point I was more used to asking for definitions and just taking shots if they refused to define them. This interview was also closest to the work I’ve been doing the past two years, but it still felt like pulling teeth talking to them. I’ve been working on these problems for a long time in the accelerator space, and thought even though this job was GPU-based I’d be fine. The accelerator world seems to define these terms more clearly with things like "fine-grained parallelism" or "coarse-grained parallelism" (like Tangram from Stanford) for how close to compute they are, or "weight stationary," "output stationary," "input stationary," or "row stationary" (Joel Emer and Vivienne Sze’s work) to describe which part of the tensor is moving. There’s even work with high-level languages or representations to define these movements (like Maestro from GATech). Coming from that space into this feels like speaking a different language where every word sounds the same and doesn’t really reflect its meaning. I am totally willing to learn all these translations, words, and definitions, but being asked to do this on the spot without prep feels insane. It felt bad that during the panel I constantly had to fight with loosely defined words or ill-posed problems that I had to react to in real time. I don’t think any actual role would have me work like this; in reality you’d sit down and understand things before simulating them. But in this process, all I got was a mix of poorly formulated problems and definitions that mostly only made sense to the people asking them. I don’t think they actually tested my ability to do the job; at best they tested if I could be as effective as them on day one, which seems unreasonable. I’d be surprised if anyone could pass the same process I went through without extra info about their language, the types of problems, clearer definitions, or some level of nepotism.

Four rounds of technical interviews with a positive feeling, mainly focusing on the candidate's projects, basic professional knowledge, and coding ability. Each interview is about one hour, and the interviewer gives feedback within one or two days. The whole interview process moves relatively fast.

2024 New grad position. First round is split into two 45min technical interviews with a hiring manager and a team lead, focusing on your relevant experiences (school projects and internships), and the fundamentals of data structure and computer architecture. Second round is with another hiring manager who also tested me on data structures and cpu/gpu knowledge. I didn't have much experience in computer architecture except that I know gpu programming and a little bit of gpu architecture. I got rejected in the end.