Both the Union and the Confederacy maintained prison camps for people captured in battle. In the second half of the war, prison overcrowding became a serious problem for both sides. Overcrowding combined with a lack of resources, especially in the Confederacy, turned the prisons into deaths camps. The most infamous prison camp was Andersonville, a Confederate prison outside Macon, Georgia which was opened in February of 1864.

For the first two years of the war, both sides exchanged prisoners at certain intervals, keeping prisons manageable in terms of size. In May of 1863, however, the Confederacy began authorizing the enslavement of African American soldiers. In reaction to this, the Union War Department soon ended prisoner exchange.

While Union prisons became nightmares of overcrowding, disease and death, the situation in the Confederacy was even worse. With the Confederacy suffering a serious lack of funds, there were few resources available for Confederate troops, leave alone Union prisoners. Thus, Confederate prisons tended to be even more poorly-kept and poorly-supported than Union prisons. In most cases, the high mortality rates of these camps was believed to be due to lack of resources, rather than any particular malicious intent. One exception to this was Andersonville Prison.

Andersonville Prison was notorious for being a place of suffering and death. Rations for prisoners were limited to half a cup of corn meal and two tablespoons of sugar a day. While some of the difficult circumstances in which prisoners lived resulted from the Confederacy's lack of resources, others were the result of deliberate cruelty. One Atlanta newspaper reported that 300 Union prisoners died at Andersonville in one hot day, and added "we thank heaven for such blessings." In outrage, many Northerners urged that Confederates in Union prisons be treated with similar cruelty. Lincoln refused to do so, however. After the war, Captain Henry Wirz, the Swiss-born commander of the camp, was tried and found guilty of war crimes. He was hanged, the only officer on either side to be executed for war crimes. Clara Barton, who later founded the American Red Cross, oversaw the creation of a cemetery for Union soldiers who had died in Andersonville.