Q: Are all atoms radioactive?

The original question was: Some elementary particles spontaneously break apart at a given rate. Can the same be said about normally stable atoms and molecules? That is, even though they are stable, does their natural internal activity lead to a certain probability that they will come apart?

Physicist: Nope!  A thing is either stable or it’s unstable.  For example, carbon 12 is stable forever.
However, some of the things that we consider to be stable are only very nearly stable.  For example, Uranium 238 has a half-life of about 4.6 billion years, which is basically forever.
The question comes down to “if this thing falls apart does it release energy?”  If it does release energy then it’s unstable, and if it doesn’t then it’s stable (or you’d have to add energy to knock it apart).

The "binding energy" per proton or neutron. Energy is gained by either combining atoms (fusion) lighter than iron, or breaking apart atoms (fission) heavier than iron. As a result, most atoms heavier than iron can fall apart on their own (they're radioactive).

Answer gravy: In an atomic nucleus there are two forces acting against each other: the Nuclear Strong force and the Electric force.  The Strong force acts on protons and neutrons and holds them together, while the Electric force acts on protons and tries to make them fly apart (“likes repel”).  Over distance the Electric force gets weaker, but it still has an infinite range.  The Strong force on the other hand has an extremely limited range.

The electric force can reach across the nucleus, but the Strong force can't. For large atoms all of the protons are working together to push the nucleus apart, but the Strong force is stuck working with just protons and neutrons that are right next to each other.

So the Electric force is working to push the nucleus apart, and it can work with every proton in the nucleus (size of an atom < ∞).  But the Strong force only works between the protons and neutrons that are basically touching.  A large atom is way too big for the Strong force to work across.  So while the force that keeps each proton or neutron in the nucleus levels out for bigger and bigger atoms, the force pushing the nucleus apart just keeps going up.

A good way to talk about forces and energies is to draw a “potential diagram”.

The strong force potential (green) and the electric potential (blue). The strong potential does nothing until you're very close then drops suddenly, while the electric potential just keeps getting higher as you get closer. Together they form a bump and then a trough. The energy ("height") difference between the bottom of the trough and the surrounding space tells you whether escaping particles will produce energy (drop down) or not. For example, the potential pictured above would be for an unstable nucleus.

The combination of the strong pull and the electric push creates a bump that, normally, would be impassible.  However, (basically) because of the uncertainty principle, parts of the nucleus can “tunnel” out through that potential wall.  If they can gain energy by sliding away then they can stay outside, and fly off as radiation and decay products.  If they find that they need energy they don’t have, then they stay where they are (in the nucleus), since energy conservation makes the “away from the atom” state impossible.

So whether or not something is stable depends on whether or not there’s a net energy gain by undergoing radioactive decay.  The stability of a particular isotope has to do with the shape and height of that “potential bump”, which varies wildly from isotope to isotope.  But in general, the heavier something is, the shorter its half-life (it’s easier for stuff to tunnel out).