Definition
Let T be a theory in the language . If M is a model of describing a set theory and N is a class of M such that is a model of T then we say that N is an inner model of T (in M).
This term inner model is sometimes used to denote models which are proper classes. While the term set model is used to denote models which are sets.
Use
Usually when one talks about inner models of a theory, the theory one is discussing is ZFC or some extension of ZFC (like ZFC+ a measurable cardinal). When no theory is mentioned, it is usually assumed your inner model is a model of ZFC. However, it is not uncommon to talk about inner models of subtheories of ZFC (like ZF or KP) as well.
Related Ideas
It was proved by Kurt Gödel that any model of ZF has a least inner model of ZF (which is also an inner model of ZFC + GCH), called the constructible universe, or L.
There is a branch of set theory called inner model theory which studies ways of constructing least inner models of theories extending ZF. Inner model theory has led to the discovery of the exact consistency strength of many important set theoretical properties.