The red brick is the part that was rebuilt by Union forces after they captured the fort.