Cloth of the brotherhood is obtained from the challenge mission, you have to survive all 7 mobs and you only get ONE cloth. Or you can do the elite dungeon (Which I was doing but having trouble defeating the boss who returns like 150% damage back, even though it says 100%)
The cloth is used to craft hero armours I believe.
Indeed it is, and it is surprising that they putted effort to upgraded hero armors (some of them are not better looking in my opinion than without upgrade).