Marko Arnautovic gave the visitors a 25th-minute lead, with Nacho Monreal restoring parity five minutes later.
An entertaining and error-strewn contest continued to flow from end to end but an injury to Arnautovic robbed West Ham of their prime attacking threat before Diop erred.
Welbeck came off the bench to gleefully blast in from close range, ensuring Manuel Pellegrini's fledgling West Ham tenure stands at three defeats out of three.
Arnautovic asked constant questions of Arsenal's high and shaky backline and had already thrashed into the side netting before he exchanged passes with Felipe Anderson to arrow a 20-yard drive into the bottom corner.
Monreal made light of an earlier knock to slam home the equaliser when West Ham failed to clear Hector Bellerin's cross.
Bellerin's mistakes at the other end remained a problem, with Sokratis Papastathopoulos forced into a desperate block from Felipe Anderson before the Spain full-back played Robert Snodgrass onside to shoot too close to Petr Cech.
Cech stood firm to thwart Arnautovic in the 55th minute and apparent knee injury he sustained during that attack proved pivotal.
Diop allowed Alexandre Lacazette too much room to turn and shoot, with ex-Arsenal goalkeeper Lukasz Fabianski beating the shot away, and the 21-year-old centre-back was the architect of his own misery 20 minutes from time.
A miscued clearance spiralled up in the air and Aaron Ramsey nodded down for Lacazette to drive into the goalmouth, with Diop only able to divert into his own net.
Felipe Anderson continued to probe, flashing agonisingly across the face of goal in injury time, meaning Arsenal nerves only settled when Welbeck swivelled to dispatch Bellerin's pass.
Arsenal arguably attacked more fluently in their 3-2 defeat to Chelsea last time out, but Emery will be delighted with a three-point haul to boost an ongoing teething process. This was West Ham's best showing under Pellegrini – clearing an admittedly low bar – but any prolonged absence for Arnautovic would represent a major problem.
Some sharper finishing from the Austria forward might have seen West Ham take all the points back across London but Arnautovic was very impressive as their focal point, revelling in Arsenal's defensive uncertainty. His goal was well taken and the 29-year-old – handed the captain's armband by Pellegrini – performed with a level of maturity that has not always been evident during a career that has thrilled and frustrated in equal measure.
Whipping the Arsenal defenders he inherited into shape will understandably take Emery time. However, it is hard to escape the impression his board landed him with an extra problem by bringing in the sluggish and untidy Sokratis, who does little to inspire confidence.
