Definition
A formal system is complete if every truth expressible in its language is provable inside the system. There are two related senses. Semantic completeness: every statement true in every model of the axioms is provable. Syntactic completeness: for every sentence A, either A or its negation ~A is provable.
Propositional logic is both semantically and syntactically complete. First-order predicate logic is semantically complete (Gödel's 1929 completeness theorem) but not syntactically complete in interesting subtheories. Arithmetic, formalized in TNT or Peano arithmetic, is incomplete in the syntactic sense: Gödel's 1931 first incompleteness theorem produced a true sentence neither it nor its negation could prove.
Why it matters
How it works
To check completeness you ask: for every well-formed sentence in the language, can the rules derive it (semantic version) or one of it/its negation (syntactic version)? For propositional logic, both can be answered yes constructively — every tautology has a syntactic proof, and every consistent set of sentences has a model.
For arithmetic, completeness fails because the language can express claims about its own provability. Gödel constructs a sentence that, decoded, says "I am not provable in this system." If the system is consistent, the sentence is true but unprovable. Completeness collapses with no remedy short of leaving the system for a strictly stronger one — which has its own incompleteness.