For matrices with displacement structure, basic operations like multiplication, inversion , and linear system solving can all be expressed in terms of the following task: evaluate the product AB, where A is a structured n × n matrix of displacement rank α, and B is an arbitrary n × α matrix. Given B and a so-called generator of A, this product is classically computed with a cost ranging from O(α^2 M (n)) to O(α^2 M (n) log(n)) arithmetic operations, depending on the type of structure of A; here, M is a cost function for polynomial multiplication. In this paper, we first generalize classical displacement operators, based on block diagonal matrices with companion diagonal blocks, and then design fast algorithms to perform the task above for this extended class of struc-tured matrices. The cost of these algorithms ranges from O(α^{ω−1} M (n)) to O(α^{ω−1} M (n) log(n)), with ω such that two n × n matrices over a field can be multiplied using O(n^ω) field operations. By combining this result with classical randomized regularization techniques, we obtain faster Las Vegas algorithms for structured inversion and linear system solving.