I've just taken a look at this Quora question and I'm surprised too at the (unusually) poor quality of the answers. I can think of two approaches to see that a system does not violate the second law of thermodynamics.

The first one is very general: You assume something about the system (Such as that the dynamics of the system [both atoms and pawl] is time-reversible) and derive a fluctuation relation, which tells you that the probability of an entropy-increasing event is always larger or equal to the probability of an entropy-decreasing event. Then you can look at every ingredient in your system to check that it satisfies the assumptions of the fluctuation theorem (For example, motion of a mass-spring system such as the pawl is time-reversible because if you watch the motion in reverse you get an equally physically valid trajectory, which is true for Hamiltonian systems in general).

https://en.wikipedia.org/wiki/Crooks_fluctuation_theorem

https://en.wikipedia.org/wiki/Microscopic_reversibility

This is a very abstract procedure but it guarantees that if your system does not contain 'naughty' elements (That don't follow Hamiltonian dynamics or break time reversal symmetry somehow), then it is not going to violate the second law. There are no such elements in the racthet-and-pawl device as it is typically drawn, with or without gas on the other side.

The second one (that is probably what you are looking for) is to simulate the system (either using a computer or mentally) and seeing what can go wrong. Looking at this drawing of a racthet and pawl system:

http://i.kinja-img.com/gawker-…/t83tjdx6klcwghgqeosb.png

Let's try a bit of mental simulation: What happens after the pawl jumps from one tooth to the next one. Option (a): The elastic potential energy in the pawl gets dissipated somehow. If that's the case, then we're done. Your system is not Hamiltonian (has energy loss) but it's dissipating energy somewhere (heating a cold reservoir) in addition of producing work. If the cold reservoir is too hot, there will be energy transfer from the reservoir to the pawl (i.e. allowing it to jump up sometimes and letting the ratchet go in reverse). If you include the dynamics of the reservoir that takes the potential energy from the pawl, then the whole system will be microscopically time-reversible and will follow the fluctuation theorems. What if there is no reservoir where the pawl's potential energy can go? Well, then after the first jump it will keep oscillating with an amplitude equal to the height of the teeth. This will allow the ratchet to move in any direction, and the whole system will not work.

The explanation that I've provided here is very simplistic. There are papers on the topic that are many pages long and provide a rigorous and detailed treatment, but this should give you a rough idea. If you are computationally inclined, it is not *too* hard to simulate a system like this one.