Preferring LDP Path instead of RSVP Path

Just trying to see if there is any  way to do this.
 
I have this topology
            LDP         LDPoRSVP    LDP
  [PE1]<---->[P1]<----------->[P2]<----->[PE2]
      |                                                        |
      |                                                        |
     +----------------------------------------------+
                              RSVP
 
Just in case diagram gets messed up in the post, I have attached it to the post in a file.
 
Currently PE1 connects to PE2 directly and has established RSVP LSP between them to stear the traffic (global and vrf) using TE.
 
I want to introduce 2 P routers in the path between PE1 and PE2. Initially, Both direct link and path via P1 and P2 will coexist for sometime. Now I want to steer the traffic over P1 and P2 routers between PE1 and PE2 as primary path and keep the direct link between PE1 and PE2 as secondary or backup path in case, the Path via P router fails. 
 
As you can see PE1 learns PE2's address via LDP and RSVP LSPs in inet.3 table but RSVP path is installed for forwarding due to protocol preference of RSVP being higher than LDP. Is there a way I can still make it prefer LDP path via P1 and P2 instead RSVP path over direct link without actually changing the protocol preference for RSVP and LDP?