Tensor renormalization group (TRG) has attractive features like the absence of sign problems and the accessibility to the thermodynamic limit, and many applications to lattice field theories have been reported so far. However it is known that the TRG has a fictitious fixed point that is called the CDL tensor and that causes less accurate numerical results. There are improved coarse-graining methods that attempt to remove the CDL structure from tensor networks. Such approaches have been shown to be beneficial on two dimensional spin systems. We discuss how to adapt the removal of the CDL structure to tensor networks including fermions, and numerical results that contain some comparisons to the plain TRG, where significant differences are found, will be shown.