Preheat oven to 400F. Salt and pepper both sides of chicken thighs.
In an ovenproof skillet over medium-high heat, melt 2 tablespoons of butter. Cook the chicken, skin-side-down. Once the chicken becomes opaque around the edges and the skin is golden brown, about 6-8 minutes, turn the chicken skin-side-up and place in the oven.
Bake uncovered until chicken is cooked through, about 10-15 minutes. Transfer chicken to a plate and set aside.
Place the skillet on the stovetop on medium heat. Add fennel bulb, onion and olives. Cook until onion is golden, about 5 minutes.
Add orzo. Cook until pasta is nutty brown, about 3 minutes. Pour in wine and cook, stirring until the liquid is evaporated, about 1 minute. Add broth ½ cup at a time, stirring constantly and letting broth absorb before adding more. Repeat until orzo is tender and broth is mostly absorbed, about 10-15 minutes.
Remove from heat and stir in lemon juice and butter. Top with fennel fronds and chicken. Serve immediately.