Preheat the oven to 375 F (190 C). Grease a 9x9-inch baking dish or similar casserole dish and set aside.
Place the peeled and chunked potatoes into a large pot and cover with cold water. Add a pinch of salt. Bring to a boil over high heat, then reduce the heat and simmer until the potatoes are tender, about 12 to 15 minutes when pierced with a fork.
Drain the potatoes well and return them to the pot. Let them sit for a minute to allow excess steam to escape so the mash won’t be watery.
Add the butter, sour cream, and half of the milk to the potatoes. Mash with a potato masher or a hand mixer until mostly smooth. Add more milk as needed to reach a creamy consistency. Season with the teaspoon of salt and 1/2 teaspoon black pepper, tasting and adjusting.
Fold in 1 cup of the shredded cheddar, most of the crumbled bacon (reserve a little for garnish), and two tablespoons of the sliced green onions. Mix until evenly distributed but not overworked.
Spoon the mashed potatoes into the prepared baking dish and smooth the top with a spatula. Sprinkle the remaining 1/2 cup cheddar evenly over the top.
In a small bowl, toss the panko breadcrumbs or crushed crackers with the olive oil or melted butter. Scatter the mixture over the cheese for a crunchy topping. If you prefer no topping, skip this step.
Bake in the preheated oven for 20 to 25 minutes, or until the casserole is bubbly and the top is golden. If you like a browner top, broil for 1 to 2 minutes at the end—watch closely.
Remove from the oven and let the casserole rest for 5 minutes. Garnish with the reserved bacon and the remaining sliced green onions before serving.