Crumby means of very poor quality; flimsy or similar to crumbs; covered with crumbs. The first meaning comes from nature of bread crumbs and how we treat them.
Crummy means ‘Miserable or wretched’: a crummy situation in the family.
When the reference is to actual crumbs, as in a crumb loaf or a bread crumbs, use crumbs.
When the reference is to ‘dirty, squalid, inferior, worthless’, use crummy.
