Lets discuss the early non-canonical Christian writings! You can find a huge collection of them here:
http://www.newadvent.org/fathers/index.html
Some important ones:
1 Clement
What is it?
1 Clement is a letter written by Clement of Rome according to tradition, who was the 3rd Bishop of Rome after Peter and Linus. The letter was addressed to the Corinthian congregation and contains general exhortations to adhere to the faith. It also contains an interesting passage comparing Christ to the mythical phoenix, and even suggests the Phoenix should be the symbol of Christianity as an example of the resurrection to come.
Epistles of Ignatius
What are they?
Letters by Ignatius of Antioch who was being taken to Rome for execution. Ignatius was Bishop of Antioch and wrote seven letters on his way to Rome, so our only insight into him is his last writings as he contemplates his martyrdom. Ignatius was a man of exceptional faith, even going so far as to tell the Roman congregation not to interfere in his martyrdom as it was what he desired.
Shepherd of Hermas
What is it?
One of the most popular early Christian writings it was actually considered inspired by a few early writers including Clement. The Shepherd of Hermas is about Hermas, the brother of Pius I, receiving revelation about the nature of the church and his moral duty as a Christian. It contains a lot of metaphor, depicting the Church as a woman who grows beautiful and also as a tower built by the stones which are the faithful. Hermas is guided to greater understanding by the Angel of Repentance, who reveals mysteries to Hermas to pass on to the faithful.
Epistle of Barnabas
What is it?
One of the best explanations of why Christianity is the fulfilment of Judaism ever written. The Epistle of Barnabas was written to explain why the Jewish covenant with God had ended and been replaced by the final covenant of Christ with all humanity. It goes into painstaking detail, breaking down prophecy in the Old Testament to explain why Christ was indeed the Messiah and why the Jews failed to uphold their covenant with God resulting in Christians replacing them as the elect.