OUTER BILLIARD AROUND A CURVILINEAR TRIANGLE WITH A FIXED DIAMETER

We consider an outer billiard around a Reulaux triangle. We prove the existence of infinitely many periodic points accumulating at infinity. To do so we construct a return map from a strip into itself and we study its properties. We also show some numerical simulations which, in particular, display heteroclinic intersections and Smale’s horseshoes. 2010 Mathematics Subject Classification: 37E30, 54H20.

1. Introduction 1.1.Definition.Outer billiards are a class of dynamical systems that has been studied by numerous authors and have attracted a substantial attention in the last two decades.
The outer billiard map is defined as follows, see Figure 1.Let Γ be a convex compact subset of an oriented plane.Given a point x outside of Γ there are two lines R(x) and L(x) passing through x and supporting Γ, that is such that R ∩ Γ and L ∩ Γ is in the boundary of Γ.Let R(x) be the one such that Γ lies on its (orientation) left.Then, assuming Γ is strictly convex, the map x → Φ(x) is given by the unique point on R(x) such that the point of contact between Γ and R(x) is the midpoint of the segment [x, Φ(x)] ⊂ R(x).We remark that R(x) = L(Φ(x)).The inverse map to an outer billiard map is just the outer billiard map with respect to the same convex shape but with the opposite orientation of the plane, so that both the contact midpoint and Φ −1 (x) lie on the line L(x).The orientation on a line tangent to Γ is such that x ≤ Φ(x) for any point x such that both x and Φ(x) lie on that line.
When Γ is not strictly convex, so the boundaries contain line segments, then the map is left undefined on the lines containing those segments.In order to define the dynamics properly (that is, to define all iterates) we remove from the plane all the preimages and images (as preimages of the inverse map) of such lines.This removes at most countably many one dimensional objects, so a set of full Lebesgue measure remains.
The dynamics of an outer billiard is invariant under affine transformations of the plane.
1.2.Known results.The dynamics of the outer billiard was studied since Neumann [10].Moser [8,9] popularized the system as a toy model for celestial mechanics.The main problem is whether all orbits are bounded.For so-called quasi-rational polytopes, boundedness of orbits was proven by Vivaldi-Shaidenko [15], Ko lodziej [7], and Gutkin-Simányi [6].For rational polytopes (with all vertices at rational points in some affine representation, in particular for triangles and squares) all orbits are periodic, but for the regular pentagon Tabachnikov [12] showed some aperiodic orbits exist.On the other hand Genin [4] showed that all orbits are bounded when the shape is a (non-quasi rational) trapezoid.
The possibility of unbounded orbits was shown by Schwartz [11] for the Penrose Kite (some orbits) and by Dolgopyat-Fayad [2] for a semidisk.
Boundedness of orbits was proven for sufficiently smooth Γ by Douady [3].For strictly convex Γ, Gutkin and Katok [5] proved the outer billiard is a twist map which preserves Lebesgue measure.In this paper we focus on a piecewise-smooth, strictly convex Γ.In particular, Lebesgue measure is conserved.
Leaving the Euclidean plane, Dogru and Tabachnikov [1] showed that all orbits are unbounded for a certain class of convex polygons in the hyperbolic plane.
For more details see an excellent short article in Wikipedia and also Tabachnikov [13,14].

The triangular family Γ t
We shall consider the outer billiard with respect to the Reuleaux triangle, a convex shape Γ = Γ 1 .It is a curvilinear, fixed diameter triangle, a member of the family Γ t , 0 ≤ t ≤ ∞ which we describe below.Let the equilateral triangle with vertices v i , i = 0, 1, 2 have the center o.The arithmetic on the indices will always be mod 3, for example by definition v 3 = v 0 .We will use the word edge for any of the sides of this triangle v i − v j , i = j.
For t ≥ 0 define the points and the (shorter) circular arcs a i centered at o i and joining the vertices v i−1 and v i+1 .For t = 0 the three centers are equal and equal to o, and the three arcs form a complete circle Γ 0 .For t = ∞ the arcs become the segments joining the vertices and the shape becomes the triangle Γ ∞ .For any t < ∞ the shape is strictly convex.The angle between the arc and the chord (the triangle side) equals arctan( √ 3/(1 + 2t)).The case of t = 0 (circle) gives arctan √ 3 = π/3, the case t = ∞ (triangle) gives arctan 0 = 0, which for t = 1 gives arctan 1/ √ 3 = π/6.In case of Γ 0 , a circle, the orbits conserve the distance from its center and the maps are rotations of co-centric circles.The rotation angle depends on the radius, namely for the radius r of Γ 0 and D In case of Γ ∞ , an equilateral triangle, the map is undefined on three half-lines each containing a side.On three regions defined by those halflines the map is a symmetry with respect to an appropriate vertex of the triangle.One can remove the countable family of lines consisting of the three lines containing the sides and all their images by the vertex symmetries, and on the remaining full measure set both the forward and backward dynamics are defined for all the iterates.All orbits are periodic.The removed lines leave a tiling of the plane into regular hexagons and regular triangles, where hexagons share sides with triangles only and triangles with hexagons only.The second iterate Φ 2 is a translation by one of the vectors 2(v i−1 − v i ), which are twice the edges of the triangle and at the same time are one of the main diagonals of the hexagon.Φ 2 leaves invariant the collection of hexagons and two collections of triangles.
2.1.The partitions V , A and the regions V , A. The outer billiard Φ on Γ t for 0 < t < ∞ is defined piecewise on six alternating regions A i and V i , where the region V i is followed by A i−1 , which is followed by V i+1 .For x ∈ A i the contact point of R(x) touches the arcs a i and the map behaves locally as a circular outer billiard around the center o i (when t = 1, o i = v i ).In V i it touches the vertex v i and the map behaves like a triangular outer billiard.The regions A and V are defined similarly for the inverse map Φ −1 .The regions V i and A i±1 are separated by the rays (half lines) T (v i , a i±1 ), where T (v i , a j ) is tangent to the arc a j at v i .The rays start at v i and go into the lower part of the line (according to the chosen orientation).There are six such rays, one from each line T .The complementary rays separate the regions V and A. Additionally each region A i has a little piece of boundary which is the arc a i .The regions for Φ and Φ −1 are images of each other by any axial symmetry of Γ t (which all have the same axes of symmetry as the triangle Γ ∞ ).Additionally the regions V i and V i are symmetric with respect to the point v i .
The directions of the rays depend on t.The angle from the side (v i−1 , v i ) to the ray T (v i , a i+1 ) varies from π/3 for t = 0, through π/6 for t = 1 to 0 for t = ∞, the similar angle for the ray T (v i , a i−1 ) varies from π/3 for t = 0 through π/2 for t = 1 to 2π/3 for t = ∞.In the extreme cases t = 0 the each region V i collapses to the ray 2.2.Middle point partition of Φ 2 : V V , V A, AV and AA.Consider the refinement of two partitions, one for Φ and one for Φ −1 .The outside of Γ t is divided by the whole lines T (v, a) into several regions: twelve of them are unbounded (in the extreme cases we count the regions collapsed to rays).In the case t = 1, due to the fact that the tangents to the arcs at the vertices are pairwise parallel, we have nine bounded regions; otherwise there are twelve.The unbounded half-strip on the vertex side of Γ 1 is an intersection of a A and an A region.The unbounded half-strip on the edge side is an intersection of a V and a V regions.The other six unbounded regions between the strips are intersections of either a V with an A or an A with a V .
We shall denote the dynamics of the outer billiard uses for the contact point first the arc a i then the vertex v i .The labels and ordering of the unbounded elements of this partition are depicted in Figure 2. Remark that there are strips V i V j and A i A j for each combination i = j but in unbounded A i V j and V i A j we must have i = j.The bounded regions are regions Figure 2. The middle point partition for the second iterate.The continuous rays are boundaries for the partition for Φ, the dashed rays are the boundaries for the partition for Φ −1 .
When t = 1, the lines defining the elements V V and AA are not parallel, but intersect in a finite point.If t < 1 the intersection point is above the vertex making the element AA bounded but adding a second unbounded component of V V .If t > 1 the intersection occurs on the edge side making the element V V bounded but adding another unbounded component of AA.From the projective geometry point of view it is not very illuminating, but we are interested in boundedness of the orbits and hence there might be some interest in studying such crossing points.
2.3.The period 3 orbit w i .The points w i = v i−1 +v i+1 −v i form an equilateral triangle (double the size of the triangle v i ), see Figure 4. We have w i ∈ V i−1 V i+1 for all t and Φ(w i ) = w i+1 thus forming a period three orbit.For t = ∞ the points w i are the centers of the period-six hexagons adjacent to Γ ∞ ; the large diagonal of the hexagon is twice the size of the edge.In the case t = 0, they lie on the circle where all points rotate by 2π/3.In the case t = 1, they lie in smaller, period-six hexagons, which are intersections of three strips with widths equal to the edges, see (1).
2.4.The map Φ 2 .By definition the point (x + Φ(x))/2 is the contact point for x on R(x).Whenever Φ 2 is well defined we have Φ , that is Φ 2 translates by double the vector between the consecutive contact points.
For Γ 1 the dynamics of Φ 2 is elegant when the middle point belongs to AV , V A and V V ; it is more subtle for AA, see Figure 3.
When Φ(x) ∈ A i V i , then the points x, (x + Φ(x))/2, v i form the rightangled triangle equal (by symmetry) to the triangle Φ(x), (x+Φ(x))/2, v i , hence the distances dist(x, v i ) = dist(Φ(x), v i ) are, by definition of Φ on V i , equal to dist(v i , Φ 2 (x)).In other words the map on A i V i is a twist rotation about the vertex v i by an angle depending on the distance, such that the chord (x, Φ 2 (x)) has length equal twice the edge.Similarly the map on V i A i is a rotation about v i .When Φ(x) ∈ V i V i+1 , then the map Φ 2 is a translation by the vector 2(v i − v i+1 ) (remark the order of indices).However when Φ(x) ∈ A i A i+1 the translation is less obvious, and the length of the translating vector is smaller than twice the diameter.

Dynamics for Γ 1
3.1.Invariant hexagons.For Γ 1 consider the partition element V i−1 V i , which is the unbounded half-strip based on the edge (v i−1 , v i ), except without the three bounded pieces.Its image under Φ, which acts on V i as a central symmetry with center v i , is a half-strip which is a translation of A i−1 A i by the vector It follows that the intersection (1) is a regular hexagon which stays in the regions V V forever as, in fact, The union of the three hexagons forms a maximal set which stays in the union of regions V forever.By symmetry it is invariant under Φ 3 , but as Φ 3 can be understood both as a composition of three translations and as a composition of three central symmetries, Φ 3 is not an identity on H, but rather a central symmetry with respect to its center.The center of the hexagon is periodic with period 3, but other points are periodic with period 6, see Figure 4.
Remark 1.The dynamics of the points very close to Γ t follows a path with possible omissions of the symbols V .Using symmetry it is possible to find periodic orbits close to Γ t of arbitrary large period.For example, consider a regular 6n-gon circumscribing the circle of radius ∆.Cut it into 6 parts, each part formed by n consecutive sides.Remove every other part and move the remaining three parts to be tangent to the arcs a i of the Reuleaux triangle (with edge length ∆) with their endpoints on the rays ov i±1 .Then by symmetry we obtain a closed curve, a 3n-gon circumscribing Γ 1 , with a tangency point in the middle of each side.By construction the 3n vertices form a periodic orbit of the outer billiard on Γ 1 .The orbit lies entirely in the union of AA regions.For n = 1 that gives another orbit of period 3 forming an equilateral triangle of sides parallel to segments (v i , v j ) and of length 2( √ 3 − 1)∆.One can similarly construct orbits which lie only in AA regions except for three points in AV and their three images in V A.
We shall prove that there are infinitely many periodic points far from Γ 1 .
Theorem 2 (Infinitely many periodic points).For every N > 0 and D > 0 there exists a point x with dist(x, Γ 1 ) > D which is periodic with period greater than N .
3.3.Induced first return map on a strip.We shall define three strips S 0 , S 1 and S 2 , such that S 2 is the image of S 1 under rotation through angle 2π  3 and S 1 is the image of S 0 under rotation through angle 2π 3 (see Figure 5).By symmetry it is enough to provide the construction for one index, namely Both strips have width ∆ and they are parallel and adjacent.We shall consider S 1 = S − ∪ S + for points sufficiently far from Γ 1 .We see that Φ 2 S − = ΦV 2 V 0 is again parallel and adjacent to S + , hence the iterates Φ 2 of S constitute a connected region, a fan of sorts turning clockwise.The first image Φ 2 (S 1 ), for points sufficiently far, lands in the region A 0 which means that for After this we are in the region Φ −1 V 0 A 0 and there Φ 2 uses the vertex v 0 and then the arc a 0 .In other words it is a rotation about v 0 .The iterates of Φ 2 will rotate a point of Φ −1 V 0 A 0 about v 0 until it enters the strip S 2 .The reason we cannot miss this strip is that its width is 2∆ which is an upper bound for the distance between x and Φ 2 (x).The angle of rotation equals 2 arcsin(∆/ dist(x, v 0 )).
We shall be interested in the map G which is the map from S 1 to S 2 defined by Φ 2k+1 (x) with minimal k = k(x), or the first entry map, or if one would identify the strips, the first return map.We will find points x ∈ S 1 and y ∈ S 2 which are symmetric (with respect to Γ 1 symmetries) with y = G(x).Such a point will be periodic as then G 3 (x) = x by symmetry.

Polar coordinates.
In order to specify the rotation angle we shall use polar coordinates (r, θ) in which r is a distance from v 2 and θ is the angle to the common boundary of S − and S + with positive angles in S − and negative in S + .The boundary of S 1 satisfies ±∆/r = sin θ, where the sign + corresponds to the boundary of ∂S − and the sign − to ∂S + .Again we remind that Φ 2 (∂S − ) ⊂ ∂S + .
The map G can be represented in the polar coordinate system.If x ∈ S 1 and y = G(x) ∈ S 2 then for polar coordinate (r, θ) of x with respect to S 1 we use the polar coordinate (d, ψ) for y with respect to S 2 .We can write d = d(r, θ) and ψ = ψ(r, θ).We introduce two decreasing sequences which represent the rotation angles, which change as the fan opens up: Theorem 3. Let (r, θ) be the polar coordinate of x ∈ S 1 and ( ) be the polar components of G(x) ∈ S 2 as above.Then (3) Identifying corresponding pairs of boundary points (r,±arcsin(∆/r)) of S 1 and (d, ± arcsin(∆/ψ)) of S 2 , the map G : S 1 → S 2 is continuous (for points far enough to fit the construction). Proof: (1) Suppose x ∈ S − has polar coordinates (r, θ), with θ > 0. Then Φ Since then, as we landed in Φ −1 V 0 A 0 , in order to get to S 2 we apply several times the rotation about v 0 which does not change this distance.
The remaining iterates are as in the case x ∈ S − .
(2) For x ∈ S − let y = Φ 2 (x), z = Φ 3 (x) and c be the intersection of the open interval (x, y) with the boundary of S + 1 .Let α be the angle at y in the triangle v 0 , y, z, β be the angle at v 0 in the triangle y, v 0 , c. Then the angle φ at v 0 in the triangle z, v 0 , c fulfils: φ = π − 2α − β as in the isosceles triangle y, v 0 , z the angle at v 0 is π − 2α.Hence the angle coordinate of z with respect to the polar coordinates of where, as we already know dist(y, v 0 ) = d.After Φ 3 (x) we continue with the rotation about v 0 by 2 arcsin(∆/d), noting that w, v 0 , Φ 2 (w) form an isosceles triangle with side lengths d and ∆, where w is a point in the trajectory of z = Φ 3 (x) under Φ 2 .
Similar calculations can be made for x ∈ S + .
(  For large r, quantities r and d are comparable, the angle θ is very small and so are ∆/r and ∆/d.Hence by approximation γ ± k (r, θ) ≈ π/3 − k2∆/d.Now we can use the inequalities defining ψ(r, θ): arcsin(−∆/r) < γ ± k (r, θ) and arcsin(−∆/r) ≥ γ k+1 k ± (r, θ).Let S denote the union S 0 ∪ S 1 ∪ S 2 .Identifying points via rotations through 2π  3 , we obtain a quotient space S and a map F = G : S → S. We retain the (r, θ) coordinates and the notation (d(r, θ), ψ(r, θ)) for the image of (r, θ) under F .We can denote by v the projection of the vertices v i .Let S + and S − also denote the corresponding strips in the quotient space.Let U k denote the region {(r, θ) : k(r, θ) = k}.Theorem 5.For r > 10∆ the fixed points of F lie on the curves {(r, ± arcsin(∆/2r)) : r > 10∆}.There is exactly one fixed point in each U k ∩ S − and in each U k ∩ S + (for all large k).

Simulations and open questions
Most of the material in this section was obtained by means of simulations.We are going to study properties of the map F : S → S defined in the previous section.We choose coordinates, identifying S with S 1 , such that v 2 = (0, 0), v 0 = (1, 0) and v 1 = (1/2, √ 3/2), so that ∆ = 1, and the strip S is the area −1 ≤ x < 1, y > 10.We know (see Theorem 5) that the map F has fixed points on the lines x = −0.5 and x = 0.5.From the derivative estimate in the proof of that theorem, for example, one can show that the occurrence of fixed points happens at (vertical) intervals of length approaching 6/π ≈ 1.91 (see also Figure 8).
On Figure 6 we see a simulation of the dynamics near a fixed point (0.5, Y ) of F , where Y 100.2680425.The point (0.5, Y ) is a saddle.We have taken a fragment of its unstable manifold and iterated it 50 times forward.The simulated picture looks like a textbook example of a homoclinic intersection -with wider and wider wiggles of the unstable manifold intersecting the stable manifold.Nevertheless this is only a simulation and the effect can be due to numeric inaccuracies.This leads us to Open Question 1. Do stable and unstable manifolds of the fixed point (0.5, Y ) and other fixed points of F on the line x = 0.5 form a homoclinic intersection or a homoclinic connection?
We suspect that the picture is correct and that the answer to Open Question 1 is 'a homoclinic intersection'.Note that if the answer to Open Question 1 were 'a homoclinic connection', it would imply that all orbits of the map F (and hence of Φ) were bounded, providing a proof of our main conjecture.Conjecture 1.All orbits of the map Φ are bounded.
We have done simulations for other points on the line x = 0.5 with larger values of y.The larger the y the smaller the "wiggles" are, suggesting that with distance the angle of intersection between the stable and unstable manifolds decreases.We think this fact could be used in a proof of Conjecture 1, to bound the rate of escape of a point.
The map F has a fixed point (−0.5, Y ) inside the contour formed by the unstable manifold of the point (0.5, Y ).The point (−0.5, Y ) has a linear part of a center type.Simulations indicate that it is a center indeed -the orbits of the points surrounding it seem to be contained in ovals, see Figure 7.The picture was obtained by taking 600 evenly distributed points on a segment (0.5, Y − 0.25), (0.5, Y + 0.25), iterating each one of them forward 400 times and plotting each of the trajectories.The final picture, Figure 8, was obtained by taking 400 evenly distributed points on a segment (0, 98), (0, 102), iterating each one of them forward 1000 times and plotting each of the trajectories.The picture is very suggestive and leads us to Conjecture 2. The strip S is filled with horizontal curvilinear strips each one of them filled with periodic points of the center type, located between separatrices of periodic points of a saddle type.

)
Continuity of G on the cylinder obtained by identifying boundary points of S follows immediately from the formulas for G (as arcsin((∆ − r sin |θ|)/d) → 0) and for ψ.

Remark 4 .
Given a point x = (r, θ) ∈ S 1 the integer k = k(x) = k(r, θ)minimising either of the expressions in the previous theorem satisfies for large r So for large r (and one can check r > 10∆ is enough), r → ψ(r) is strictly monotone (and continuous) except at discontinuities, so ψ(I k ) = (− arcsin(∆/r), arcsin(∆/r)].Let r 0 be the unique point in I k with ψ(r 0 ) = arcsin(∆/2r).Then p(r 0 ) is the unique fixed point of G in U k ∩ S − .If p(r) ∈ S + , we similarly derive thatDψ(r) = −π/3r + O(1/r 2 ),and that there is a unique fixed point in each U k ∩ S + .

Figure 6 .
Figure 6.Unstable manifold of the fixed point (0.5, Y ) of F , with Y 100.2680425.