True etching means the surface of the acrylic is eaten off, and textured. Dremel, laser, or chemical will do it. Whatever happens, the frosting must go first.
The usual paste for glass etching is called "Armour Etch" and available at craft or glass shops. But I checked and Armour Etch does not etch acrylic, only glass. Every town has a few glass shops that sell custom cut panes... not very expensive... something to consider.
If your design is all lines without solid etched areas, and you don't mind a freehand look, then a toothed metal Dremel bit is probably the way to go. This will plough narrow grooves into the acrylic. You could even keep the frost while you work through it.
You could order a new window, etched by a laser etching service.
I couldn't find any chemical products for etching acrylic. But that only means nobody sells anything competitive with common solvents that etch, or cloud, acrylic. Acetone is a solvent that etches acrylic nicely. It's actually used to weld acrylic. It won't eat right through, but it gives acrylic a porous, finely textured, white surface. You'll need to make a cream of this viscous liquid though so it's workable, and doesn't run under your masking. Mix it with flour I guess. Acetone should be available anywhere paints are sold, and it's cheap.