Tensor networks, and in particular Projected Entangled Pair States (PEPS), are a powerful tool for the study of quantum many body physics, thanks to both their built-in ability of classifying and studying symmetries, and the efficient numerical calculations they allow. In this work, we introduce a way to extend the set of symmetric PEPS in order to include local gauge invariance and investigate lattice gauge theories with fermionic matter. To this purpose, we provide as a case study and first example, the construction of a fermionic PEPS, based on Gaussian schemes, invariant under both global and local U(1) gauge transformations. The obtained states correspond to a truncated U(1) lattice gauge theory in 2+1 dimensions, involving both the gauge field and fermionic matter. For the global symmetry (pure fermionic) case, these PEPS can be studied in terms of spinless fermions subject to a p-wave superconducting pairing. For the local symmetry (fermions and gauge fields) case, we find confined and deconfined phases in the pure gauge limit, and we discuss the screening properties of the phases arising in the presence of dynamical matter.