Coexistence of Elastic Modulations in the Charge Density Wave State of 2H-NbSe2

Bulk and single-layer 2 H-NbSe2 exhibit identical charge density wave order (CDW) with a quasi-commensurate 3 × 3 superlattice periodicity. Here we combine scanning tunnelling microscopy (STM) imaging at T = 1 K of 2 H-NbSe2 with first-principles density functional theory (DFT) calculations to investigate the structural atomic rearrangement of this CDW phase. Our calculations for single-layers reveal that six different atomic structures are compatible with the 3 × 3 CDW distortion, although all of them lie on a very narrow energy range of at most 3 meV per formula unit, suggesting the coexistence of such structures. Our atomically resolved STM images of bulk 2 H-NbSe2 unambiguously confirm this by identifying two of these structures. Remarkably, these structures differ from the X-ray crystal structure reported for the bulk 3 × 3 CDW which in fact is also one of the six DFT structures located for the single-layer. Our calculations also show that due to the minute energy difference between the different phases, the ground state of the 3 × 3 CDW could be extremely sensitive to doping, external strain or internal pressure within the crystal. The presence of multiphase CDW order in 2 H-NbSe2 may provide further understanding of its low temperature state and the competition between different instabilities.

Abstract. Bulk and single-layer 2H-NbSe 2 exhibit identical charge density wave order (CDW) with a quasi-commensurate 3×3 superlattice periodicity. Here we combine scanning tunnelling microscopy (STM) imaging at T = 1 K of 2H-NbSe 2 with firstprinciples density functional theory (DFT) calculations to investigate the structural atomic rearrangement of this CDW phase. Our calculations for single-layers reveal that six different atomic structures are compatible with the 3×3 CDW distortion, although all of them lie on a very narrow energy range of at most 3 meV per formula unit, suggesting the coexistence of such structures. Our atomically resolved STM images of bulk 2H-NbSe 2 unambiguously confirm this by identifying two of these structures. Remarkably, these structures differ from the X-ray crystal structure reported for the bulk 3×3 CDW which, in fact, is also one of the six DFT structures located for the single-layer. Our calculations also show that due to the minute energy difference between the different phases, the ground state of the 3×3 CDW could be extremely sensitive to doping, external strain or internal pressure within the crystal. The presence of multi-phase CDW order in 2H-NbSe 2 may provide further understanding of its low temperature state and the competition between different instabilities.

Main text.
The genuine origin of the charge density wave (CDW) state in NbSe 2 has been a matter of continuous debate. 1,2 Clearing up this point is an unavoidable issue in any attempt to understand the interplay between CDW and superconducting (SC) states in this paradigmatic material. 3 Bulk 2H-NbSe 2 is a room temperature metal which at 33 K undergoes a transition towards an atypical CDW state 4 with practically no resistivity change through the transition. Below 33 K the system exhibits a modulated almost quasicommensurate 3×3 structure 5 and at 7 K enters into a SC state 6 . It has been recently shown that the CDW order remains intact in single-layer NbSe 2 7 . In contrast with 2H-NbSe 2 , the CDW modulation in bulk 2H-TaSe 2 is 3×3 commensurate at low temperature. 5 Dichalcogenides of the 2H-MX 2 family provide promising playground for the study of 3 competing electronic instabilities in the 2D limit, 2H-NbSe 2 being specially challenging because of the incommensurability of its CDW.
Single-layers of 2H-NbSe 2 (from now on we will refer to them simply as NbSe 2 ) are hexagonal layers of Nb atoms in a trigonal prismatic environment of Se atoms (Figure 1a). 8 Although the superlattice periodicity (quasi-commensurate 3x3) has been accurately measured, 5 the microscopic structure of the elastic distortion that accompanies the CDW phase still remains elusive. The layers of bulk 2H-NbSe 2 were recently found to exhibit a continuous pattern of overlapping star-shaped Nb atom clusters extending along the layer. 9 More recently, a first-principles DFT study on single-layer NbSe 2 assuming a commensurate 3×3 modulation revealed that several structures with nearly equal stability but different distortion patterns are compatible with a 3×3 modulation. 10 This suggests a very flat potential energy surface and a plausible coexistence of the different modulations.
This potentiality apparently stimulated two subsequent theoretical works. 11 evolution with doping was also followed. While some of the phases were found for the whole range of doping considered, others were observed for some doping intervals. Doping is therefore used as a practical way to unravel competing structures in our calculations.
Nevertheless, we note that our results of the stability of the several structures as a function 4 of doping can also be relevant to experiments where injection of carriers in the single layers is achieved through electric field gating. 13 We also note again that our commensurate 3×3 structural models are only (close) approximations to the true experimental incommensurate structures. Full optimization of the non-modulated structure of single-layer NbSe 2 leads to a lattice constant of 3.48 Å, in good agreement with plane-wave type DFT studies, 11 3.47 Å, and the experimental value for the bulk, 3.44 Å. 8 The band structure and Fermi surface of single-layer NbSe 2 in both modulated [10][11][12] and non modulated [14][15][16] structures have been discussed before. Such Fermi surface contains rounded hexagons and rounded triangles centered at Γ and K points of the Brillouin zone (BZ), respectively. The calculated Lindhard response function for the optimized non modulated single-layer NbSe 2 is shown in Figure 1b. As it occurs for the bulk, 17 there are no sharp maxima around the (a*/3, 0) point and equivalent ones of the BZ which could justify a Fermi surface nesting driven mechanism of the CDW, but a very shallow region around the Γ -M direction. In contrast, as can be seen in the phonon band structure of the optimized non modulated structure shown in Figure 1c, one of the phonon branches becomes clearly unstable in a large part of the -M segment of the BZ, with a maximum imaginary frequency near but not exactly at the a*/3 point. The presence of phonons with imaginary frequency around this point indicates that the system is unstable with respect to incommensurate distortions with a periodicity not far from 3×1 (and symmetry equivalent). The combination of the three equivalent distortions (threefold symmetry, i.e. triple-q or 3Q mechanism), leads to an incommensurate structure close to 3×3. We conclude that the modulation of the single-layer NbSe 2 is a strong-coupling CDW caused by electron-phonon coupling, as it has been proposed for the bulk 16 and we previously discussed for single-layers. 10 We then performed structural optimizations imposing a 3×3 periodicity for the pristine as well as for several doping levels of single-layer NbSe 2 . We also checked that the phonon instability remains under doping: as we show in the Supplementary Information, the effect of doping does not change this picture qualitatively, although the precise shape of the unstable phonon branch and the position of the minimum experience some small changes.
A summary of this study is reported in Figure 2. Up to six different modulations compatible with a 3×3 cell were found. One of the structures exhibits centered hexagonal clusters of Nb atoms and single Nb atoms in between (noted Hexagons in Figure 2). Another structure contains a continuous pattern of overlapping star-shaped Nb atom clusters (noted Stars in Remarkably, for most doping levels the energy difference between these structures is extremely small (between fractions of a meV and at most 2 meV) so that it is likely that some of the structures experimentally coexist. Let us stress that our calculations are carried out for a commensurate 3×3 CDW structure but the real modulation is incommensurate. As a consequence of this fact and the very small energy differences, the data in Figure 2a should be only taken as suggesting that some of these structures may coexist in real samples and weak changes in doping, strain or internal pressure of the crystal may alter such coexistence.  Figure 3). 18 As a first result, the experimental phase labelled in blue can be 9 identified with the theoretical T2' . Figures 3g and 3i show the simulated STM images of this phase at ± 0.05 V that compare with the experimental STM images in Figures 3c and   3e, respectively. The theoretical 3×3 unit cells reproduce the relative intensity of the Se atoms within the unit cells simultaneously for both polarities as well as their relative orientation. Regarding the yellow phase, it can be identified with either the T1 or T2 phases since they exhibit almost the same patterns at both polarities and, therefore, are practically indistinguishable. Figures 3h and 3j show the simulated STM images for the T1 phase at ± 0.05 V (those for T2 are almost identical) for comparison with the experimental ones in Figures 3d and 3f, respectively. Here, again, the patterns within the 3×3 unit cells show a good agreement between theory and experiment. We therefore assign the observed yellow (blue) phases to the T2' and T1/T2 phases. Although this is the most likely correspondence between the experimental and theoretical phases, the blue phase can also be identified as the Stars phase according to the superlattice. However, this would imply a rotation of 180° of the crystal lattice with respect to the T1/T2 phase and, therefore, their mutual coexistence is not compatible in light of the experimental data.
In summary, on the basis of first-principles DFT calculations six different structures are found to be compatible with the 3×3 CDW structure of NbSe 2 . All these structures are found to coexist in a very narrow energy range of 2-3 meV. Their relative stability can be subtly altered by doping or strain. Imaging the surface of bulk 2H-NbSe 2 with atomic resolution allowed us to identify two of these structures, as anticipated by our theoretical simulations. Intriguingly, these structures differ from the X-ray crystal structure reported for the bulk 3×3 CDW 9 which, in fact, is also one of the six DFT structures located for the single-layer (Stars in Figure 2). Preliminary calculations for slabs with a different number of layers suggest that the actual structure stabilized may change from layer to layer, i.e., the energetic preference may depend also on the internal pressure.  Computational details. The geometrical optimizations, electronic and phonon band structures were carried out with density functional theory (DFT) 19,20 using a numerical atomic orbitals approach implemented in the SIESTA code. 21,22 We used the Perdew-Burke-Ernzerhof (PBE) functional to account for the exchange-correlation energy. 23 A splitvalence double-ζ basis set 24 was used to describe the valence electrons wave function, while the core electrons were replaced by norm conserving scalar relativistic pseudopotentials 25 factorized in the Kleinman-Bylander form. 26 The 4p shell of Nb was included in the valence explicitly as semicore states. For a good description of the free standing layer, we placed the single layer in a vacuum space of 50 Å to avoid interactions between the layer and its images. We used an energy cutoff of 2500 Ry for the real space integration. A tolerance of 10 −5 and 10 −4 on the density matrix and total energy, respectively, was used in

Notes
The authors declare no competing financial interest.