Seeing as I have an acute allergy to censorship, I will publish my last comment here, as long with anything else Marco chooses to censor. If you are interested in the nature of the arrow of time, you may want to head over to The Gauge Connection for the discussion, and then come back to read my missing comment (below). You can decide for yourself who is right.

For more information, The Reference Frame has a good post on the subject. I am also working on a series of posts that I hope will demystify the second law of thermodynamics, entropy, and the arrow of time.

Here is my missing comment:

>> Of course, even if we are physicists (are you?),

Yes.

>> definitions are important. For one reason, when one goes to do measurements, vagueness can be of no help and our main tool is mathematics. Sloppiness should be rejected on any ground.

Was I vague? Sloppy? I defined exactly a test for whether or not there is an arrow of time. You can do this test in a laboratory, and you can do it in a simulation.

The test is experimental, not theoretical, so the only mathematics you need is the ability to count. Before we try to build a theory, we should have some measurements that we are trying to explain, no? We are talking about some physical phenomenon here, right? Or do you want to have a philosophical discussion?

It’s kind of funny that earlier you lectured me about what is science, and now you make this strange claim that seems to suggest mathematics comes before experiment…

You keep talking about the arrow of time, yet you refuse to discuss what is or is not an example of an arrow of time. You say we are drowning in definitions, so I suggest a simple test. You don’t even say whether you accept this test; you wave it away claiming it’s `vague’, while clearly the opposite is true. And yet, you refuse to provide a definition, an example, or a test of your own for what is the arrow of time… Why are you trying to keep this discussion on a philosophical level, instead of actually drilling down to the heart of the matter?

>> I do not need to do your simulation.

Again, you are ignoring my simple question. Here it is again: What is the result of the test I suggested? Will you not even grant me the answer to this simple question?

I am not yet trying to draw any conclusions, I’m just asking — what is the result of the experiment? What is the result of the simulation?

>> I take two liquids both in equilibrium

Well no, not really. I am taking two liquids in a specific microstate. This is not an equilibrium. I am repeating the experiment many times, but each time I am starting in a specific microstate. Not in equilibrium.

Talking about equilibrium is already trying to model the system using statistical mechanics. But the confusion lies in the transition from deterministic mechanics to statistical mechanics, so I am not using statistical mechanics — I’m sticking to the most basic things.

>> and I make them mix.

No. I let them evolve according to Newton’s laws. We want to see whether they mix or not, and this is why we do the experiment.

>> You can change these distributions as you like, making them unphysical if you want, but the problem will remain.

Again, what is this `problem’? All I did was suggest an experiment and a simulation. I haven’t drawn any conclusions. How can there already be a problem?

>> The question should be: Who puts such deterministic systems with such initial probability distributions?

What do you mean by `who puts’? I am trying to learn how a deterministic system evolves in time. I say: If the laws of nature are deterministic, and I do this experiment, what will be the result? I am not claiming that this describes nature.

>> In conclusion, you introduce an arrow of time since the start and you are a step below Boltzmann.

How did I introduce an arrow of time? Does determinism introduce an arrow of time? Does the distribution for the initial conditions introduce it? Please explain.

PDE Primer

In case you’re not familiar with the terminology, I’ll first explain what a PDE is. A simple equation contains one or more unknowns which represent numbers. For example: . An ordinary differential equation (ODE) is similar, except the unknown is a function rather than a number. Such an equation involves derivatives of the function. Here is an example:

-
The solution of this particular equation is , where can be any number. Finally, a partial differential equation (PDE) involves a function that has two or more parameters, and includes partial derivatives of this function. For example, the following equation describes waves propagating through a medium:

-
PDEs are very important in physics. In fact, many of the basic laws of nature are described as PDEs. Examples include Maxwell’s equations, Shcrodinger’s equation, and Einstein’s field equations.

On to the simulation!

Pitfall 1: Exploding Waves

So, I built my model for the problem, derived the equations, and was ready to solve them. By ‘solving’ I mean that I start out with the known function at time t=0, and I want to find out what that function is at a later time. My function initially looked like this:

Some thousands of time-steps later, it evolved into this:

So far so good, but then it completely exploded:

Going back a bit in time, I was able to trace the beginning of this explosion:

And zooming in on the ‘wavy’ part:

It looked as though waves were forming on my function, and then ‘exploding’.

Inherent Instabilities

I was certain I had a bug, but I couldn’t find it. While debugging, at one point I decreased the spatial resolution — using less points per unit of space to describe the function… and the problem was gone! So, decreasing the accuracy of my solution actually solved the instability… That was very weird.

Mentioning this to a Ph.D student at the lab, he said this problem sounded familiar to him. And as it turns out, this is a universal problem with PDEs: If the time step is too large compared with the spatial resolution, the amplitude of small waves with short wavelengths quickly increases with time until they dominate the solution. This is due to the way numerical derivatives are calculated. The difficulty here is that the time step needs to be incredibly small, making calculation unfeasible. For some equation, the situation is even worse, as they are unstable for any time step, no matter how small.

For simple PDEs, it is very easy to see this effect by taking the function f to be a wave, and watching what happens to the amplitude over time. You can see a derivation of this result here. For a more in-depth discussion, Numerical Recipes is your friend. This method of analyzing equations is called von Neumann stability analysis.

In Comes Lax

Okay, so I found out not alone, but what can be done to solve this problem? The first thing I tried was to calculate the derivative more accurately. There is a method called Savitzky-Golay, where you fit a polynomial to your function at each point, and calculate the polynomial’s derivative at that point. The brilliant thing is that this whole operation (fit + derive) can be done using a single convolution, which costs a meager O(n log n) of processing time.

So I implemented S-G, only to discover it doesn’t solve the problem. More on that in a future post.

As it turns out, there is an incredibly simple solution due to Lax, which says the following. When advancing the function value to the next time step, you do something like this for each position:

-
The Lax method says that the f[j] at the right-hand side should be replaced by an average of it’s neighboring cells:

-
And that’s it! This replacement causes a numerical diffusion that ‘sedates’ the unruly waves, causing them to decay instead of explode. The time step used in the simulation still needs to be below some value, but now it decreases linearly with the spatial distance dx, which is much better than before. So Lax saved the day — and that was the end of my first pitfall. This is getting to be quite a long post, so I’ll describe the second problem in another post. Cheers!

The solution: First, let’s show that there’s a multiple of p that is written as a series of 1′s, followed by a series of 0′s (e.g. 1111000). Consider the set of numbers {1, 11, 111, 1111, …}. For each item, calculate its ‘modulu p’: {1 mod p, 11 mod p, …}. Each of these is an integer between 0 and p-1. Because the original set is infinite, it must therefore contain two different items that are the same mod p (pigeonhole principle). Let’s call them a and b and assume a>b. Then (a-b) is the desired number, because:
(a-b mod p) = (a mod p) – (b mod p) = 0.

Okay, so for the prime p we have a number x that’s divisable by p, and:

x = 111…10…0 = 111…1 * 10^n

We can divide x by 10^n to get rid of the zeros and get y = x / 10^n = 111…1. y is still divisable by p because p isn’t 2 or 5.