Complex systems ranging from societies to ecological communities and power grids may be viewed as networks of connected elements. Such systems can go through critical transitions driven by an avalanche of contagious change. Here we ask, where in a complex network such a systemic shift is most likely to start. Intuitively, a central node seems the most likely source of such change. Indeed, topological studies suggest that central nodes can be the Achilles heel for attacks. We argue that the opposite is true for the class of networks in which all nodes tend to follow the state of their neighbors, a category we call two-way pull networks. In this case, a well-connected central node is an unlikely starting point of a systemic shift due to the buffering effect of connected neighbors. As a result, change is most likely to cascade through the network if it spreads first among relatively poorly connected nodes in the periphery. The probability of such initial spread is highest when the perturbation starts from intermediately connected nodes at the periphery, or more specifically, nodes with intermediate degree and relatively low closeness centrality. Our finding is consistent with empirical observations on social innovation, and may be relevant to topics as different as the sources of originality of art, collapse of financial and ecological networks and the onset of psychiatric disorders.