Can I put text in a box with a button?

Yes. Open the page, open the Rich Text block, and set Show As to Box. The text then sits in a rounded panel in your theme's panel color, the same panel a Call to action block uses, in the same column as the rest of your text. Add a Button in the same block to put one button under the text, with its words and where it links. Text, the default, keeps the words on the page with no panel.