I spent so long faffing around at Rakan-ji that I didn’t have much time in Kitsuki, which was a shame. It’s an old samurai town with a castle and an interesting old district built on small hills with famous slopes and stairways. Pick up a town map from a rack in the station.
Kitsuki is on the same train line as Nakatsu. There are a couple of trains an hour from Oita (an express, which takes 20-25 minutes and costs 1,580 yen, and a local train which takes 35-40 minutes and costs 650 yen. Both are covered by the Rail Pass) but there are fewer direct trains connecting Nakatsu and Kitsuki. Kitsuki train station is several kilometres out of the city centre and, as seems typical for the Kunisaki peninsular, the buses aren’t very regular.
There’s an excellent site called Bus Navi Oita that generates timetables for stops you want to travel between. Here’s a link to timetables between Kitsuki station and Kitsuki bus terminal (all sights are walkable from the bus terminal) and timetables from Kitsuki bus terminal back to the station. You can easily be waiting an hour for the next bus, so time your arrival carefully, particularly in the afternoon when it goes down to about one bus an hour. The bus ride takes 10 minutes and costs about 300 yen each way. If you really screw up the timing you could always catch a taxi, there are plenty around the station (for obvious reasons).