The assumption is that the floating rate will reset to the market rate, and that the market rate is the appropriate discount rate for the swap payments. Given those assumptions, the value of the floating-rate bond used in the swap equation will reset to par at the next coupon date. That (par) value is the present value of all future coupon plus principal payments, so the value of the floating leg does, in fact, include the aggregate of all payments; it simply does so sneakily.
@S2000magician Thanks so much for the response magician! I went through some mental gymnastics and I can see how par value actually captures the present value of all future coupon + principal payments!