Verse 4: "Then Jesus asked them, "Which is lawful on the Sabbath, to do good or to do evil, to save life or to kill?" But they remained silent" A useful verse to remember sometimes when tempted by legalistic thoughts. It will always be lawful to do good.
Verse 5: "He looked around at them in anger and, deeply distressed at their stubborn hearts....."
It's not wrong to be angry, and even deeply distressed. I think it's probably wrong if you are angry but not distressed, because that anger is probably on your own behalf, as opposed to anger + distress, which i see as the anger at someone doing wrong accompanied by a distress for that person who is lost.
