Over the last years, Projected Entangled Pair States have demonstrated great power for the study of many body systems, as they naturally describe ground states of gapped many body Hamiltonians, and suggest a constructive way to encode and classify their symmetries. The PEPS study is not only limited to global symmetries, but has also been extended and applied for local symmetries, allowing to use them for the description of states in lattice gauge theories. In this paper we discuss PEPS with a local, SU(2) gauge symmetry, and demonstrate the use of PEPS features and techniques for the study of a simple family of many body states with a non-Abelian gauge symmetry. We present, in particular, the construction of fermionic PEPS able to describe both two-color fermionic matter and the degrees of freedom of an SU(2) gauge field with a suitable truncation.