You need to make it abundantly clear that even though you are at home, you are still working.
See if your husband can try to schedule any appointments he may need on those days, if possible. At least he would have somewhere to go (and be out of your hair) for a few hours.
I understand that on his day off, he is going to want to relax, but make it clear that as much as you want him to enjoy his day off, he can't be disruptive in the same area where you are trying to work.
[
add
]
[
|
|
...
]