In the innermost regions of protoplanerary discs, the solid-to-gas ratio can be increased considerably by a number of processes, including photoevaporative and particle drift. MHD disc models also suggest the existence of a dead-zone at $R\lesssim 10$ AU, where the regions close to the midplane remain laminar. In this context, we use two-fluid hydrodynamical simulations to study the interaction between a low-mass planet ($\sim 1.7 \;{\rm M_\oplus}$) on a fixed orbit and an inviscid pebble-rich disc with solid-to-gas ratio $\epsilon\ge 0.5$. For pebbles with Stokes numbers St=0.1, 0.5, multiple dusty vortices are formed through the Rossby Wave Instability at the planet separatrix. Effects due to gas drag then lead to a strong enhancement in the solid-to-gas ratio, which can increase by a factor of $\sim 10^3$ for marginally coupled particles with St=0.5. As in streaming instabilities, pebble clumps reorganize into filaments that may plausibly collapse to form planetesimals. When the planet is allowed to migrate in a MMSN disc, the vortex instability is delayed due to migration but sets in once inward migration stops due a strong positive pebble torque. Again, particle filaments evolving in a gap are formed in the disc while the planet undergoes an episode of outward migration. Our results suggest that vortex instabilities triggered by low-mass planets could play an important role in forming planetesimals in pebble-rich, inviscid discs, and may significantly modify the migration of low-mass planets. They also imply that planetary dust gaps may not necessarily contain planets if these migrated away.