You should try to clean up your shop area after a day of potting, but that doesn't always happen.
When you do get around to cleaning, don't sweep that dry floor. You will stir up tiny particles of silica that are found in all our clay material, not to mention whatever glaze or oxide you might have spilled. This is not so good to breath.
Clean all your surfaces using a wet method. Water works fine. Give tables, chairs, etc. a good going over with a damp rag. Rinse it often. When doing the floor, I like to use a wet cotton mop to get up the bigger stuff and since I have a concrete floor, I can hose out the shop occasionally.
