JavaScript Waterfall Pattern — Sequential Async Execution

When async tasks depend on each other, you can’t run them in parallel. You need each step to complete before the next begins, passing data down the chain like water flowing down a waterfall.

This pattern appears constantly in real applications: fetch a user, then fetch their orders, then fetch payment details for each order. Each step needs the previous step’s result.

The Problem: Dependent Async Operations

Consider this scenario:

// These operations depend on each other:
// 1. Get user ID from session
// 2. Fetch user profile using that ID
// 3. Fetch user's orders using the profile
// 4. Calculate order statistics

// You CAN'T do this:
const userId = getUserId();        // async
const profile = getProfile(userId); // needs userId first
const orders = getOrders(profile);  // needs profile first

Each operation needs the result of the previous one. This is where the waterfall pattern comes in.

Callback Hell: The Old Way

Before Promises, callbacks created deeply nested “pyramid of doom”:

getUserId((err, userId) => {
  if (err) return handleError(err);

  getProfile(userId, (err, profile) => {
    if (err) return handleError(err);

    getOrders(profile.id, (err, orders) => {
      if (err) return handleError(err);

      calculateStats(orders, (err, stats) => {
        if (err) return handleError(err);

        console.log('Stats:', stats);
        // More nesting? This gets unmanageable fast.
      });
    });
  });
});

Problems with this approach:

• Deep nesting makes code hard to read
• Error handling is repetitive
• Adding/removing steps is error-prone
• Difficult to test individual steps

The async.waterfall Solution

The async library (still used in many Node.js projects) provides a clean waterfall function:

import async from 'async';

async.waterfall([
  // Step 1: Get user ID
  function(callback) {
    getUserId((err, userId) => {
      callback(err, userId);
    });
  },

  // Step 2: Get profile (receives userId from step 1)
  function(userId, callback) {
    getProfile(userId, (err, profile) => {
      callback(err, profile);
    });
  },

  // Step 3: Get orders (receives profile from step 2)
  function(profile, callback) {
    getOrders(profile.id, (err, orders) => {
      callback(err, orders);
    });
  },

  // Step 4: Calculate stats (receives orders from step 3)
  function(orders, callback) {
    calculateStats(orders, (err, stats) => {
      callback(err, stats);
    });
  }
], function(err, stats) {
  // Final callback - receives result from last step
  if (err) {
    return console.error('Pipeline failed:', err);
  }
  console.log('Stats:', stats);
});

How it works:

1. Each function receives results from the previous function
2. Call callback(null, result) to pass data to the next step
3. Call callback(err) to short-circuit and jump to the final callback
4. The final callback receives either the error or the last step’s result

Modern Approach: Promise Chains

With Promises, the waterfall pattern becomes cleaner:

getUserId()
  .then(userId => getProfile(userId))
  .then(profile => getOrders(profile.id))
  .then(orders => calculateStats(orders))
  .then(stats => {
    console.log('Stats:', stats);
  })
  .catch(err => {
    console.error('Pipeline failed:', err);
  });

Each .then() receives the resolved value from the previous Promise and returns a new Promise for the next step.

Passing Multiple Values Between Steps

Sometimes you need data from earlier steps, not just the immediate previous one:

getUserId()
  .then(userId => {
    return getProfile(userId).then(profile => ({ userId, profile }));
  })
  .then(({ userId, profile }) => {
    return getOrders(profile.id).then(orders => ({ userId, profile, orders }));
  })
  .then(({ userId, profile, orders }) => {
    // Now you have access to all previous results
    return calculateStats(orders, profile.tier);
  })
  .then(stats => console.log(stats))
  .catch(console.error);

Cleanest Approach: async/await

Async/await makes waterfall patterns read like synchronous code:

async function processUser() {
  try {
    const userId = await getUserId();
    const profile = await getProfile(userId);
    const orders = await getOrders(profile.id);
    const stats = await calculateStats(orders);

    console.log('Stats:', stats);
    return stats;
  } catch (err) {
    console.error('Pipeline failed:', err);
    throw err;
  }
}

Advantages:

• Reads top-to-bottom like synchronous code
• All previous variables remain in scope
• Single try/catch handles all errors
• Easy to add, remove, or reorder steps

Conditional Steps

Async/await makes conditional execution trivial:

async function processUser(options = {}) {
  const userId = await getUserId();
  const profile = await getProfile(userId);

  // Conditional step
  if (profile.isPremium) {
    await syncPremiumFeatures(profile);
  }

  const orders = await getOrders(profile.id);

  // Another conditional
  const stats = options.detailed
    ? await calculateDetailedStats(orders)
    : await calculateBasicStats(orders);

  return stats;
}

Building a Reusable Waterfall Utility

Here’s a utility function that runs async functions in sequence:

async function waterfall(tasks, initialValue) {
  let result = initialValue;

  for (const task of tasks) {
    result = await task(result);
  }

  return result;
}

// Usage
const stats = await waterfall([
  () => getUserId(),
  (userId) => getProfile(userId),
  (profile) => getOrders(profile.id),
  (orders) => calculateStats(orders),
]);

With Error Context

Add context to errors for better debugging:

async function waterfall(tasks) {
  let result;

  for (let i = 0; i < tasks.length; i++) {
    const { name, fn } = tasks[i];
    try {
      result = await fn(result);
    } catch (err) {
      err.step = name;
      err.stepIndex = i;
      throw err;
    }
  }

  return result;
}

// Usage with named steps
try {
  const stats = await waterfall([
    { name: 'getUserId', fn: () => getUserId() },
    { name: 'getProfile', fn: (id) => getProfile(id) },
    { name: 'getOrders', fn: (profile) => getOrders(profile.id) },
    { name: 'calculateStats', fn: (orders) => calculateStats(orders) },
  ]);
} catch (err) {
  console.error(`Failed at step "${err.step}" (index ${err.stepIndex}):`, err.message);
}

Real-World Example: API Request Pipeline

A common use case—processing an API request with multiple dependent operations:

async function createOrder(req, res) {
  try {
    // Step 1: Validate session
    const session = await validateSession(req.cookies.sessionId);

    // Step 2: Get user with permissions
    const user = await getUserWithPermissions(session.userId);

    // Step 3: Validate cart items are in stock
    const cart = await validateCartStock(req.body.cartId);

    // Step 4: Calculate pricing (uses user tier for discounts)
    const pricing = await calculatePricing(cart.items, user.tier);

    // Step 5: Process payment
    const payment = await processPayment({
      userId: user.id,
      amount: pricing.total,
      method: req.body.paymentMethod,
    });

    // Step 6: Create order record
    const order = await createOrderRecord({
      userId: user.id,
      items: cart.items,
      pricing,
      paymentId: payment.id,
    });

    // Step 7: Send confirmation
    await sendOrderConfirmation(user.email, order);

    res.json({ success: true, orderId: order.id });

  } catch (err) {
    console.error('Order creation failed:', err);
    res.status(500).json({ error: err.message });
  }
}

Waterfall vs. Parallel: Know the Difference

Use waterfall when tasks depend on each other:

// WATERFALL: Each step needs the previous result
async function getOrderDetails(orderId) {
  const order = await getOrder(orderId);           // Need order first
  const user = await getUser(order.userId);        // Need order.userId
  const payments = await getPayments(order.id);    // Need order.id

  return { order, user, payments };
}

Use parallel when tasks are independent:

// PARALLEL: Tasks don't depend on each other
async function getDashboardData(userId) {
  const [profile, notifications, recentOrders] = await Promise.all([
    getProfile(userId),
    getNotifications(userId),
    getRecentOrders(userId),
  ]);

  return { profile, notifications, recentOrders };
}

Combine both when appropriate:

async function getFullOrderDetails(orderId) {
  // First: get the order (needed for subsequent calls)
  const order = await getOrder(orderId);

  // Then: fetch related data in parallel (all need order, but not each other)
  const [user, payments, shipment] = await Promise.all([
    getUser(order.userId),
    getPayments(order.id),
    getShipment(order.shipmentId),
  ]);

  return { order, user, payments, shipment };
}

Common Pitfalls

1. Forgetting to await

// Bug: getProfile returns a Promise, not the profile
async function broken() {
  const userId = await getUserId();
  const profile = getProfile(userId);  // Missing await!
  console.log(profile.name);           // undefined - profile is a Promise
}

2. Unnecessary Sequential Execution

// Slow: these don't depend on each other
async function slow() {
  const users = await getUsers();      // Wait...
  const products = await getProducts(); // Wait... (but why?)
  const orders = await getOrders();     // Wait... (unnecessary)
}

// Fast: run independent operations in parallel
async function fast() {
  const [users, products, orders] = await Promise.all([
    getUsers(),
    getProducts(),
    getOrders(),
  ]);
}

3. Swallowing Errors

// Bad: error disappears
async function bad() {
  try {
    const result = await riskyOperation();
  } catch (err) {
    console.log('Something went wrong'); // Error details lost
  }
}

// Good: preserve error information
async function good() {
  try {
    const result = await riskyOperation();
  } catch (err) {
    console.error('Operation failed:', err);
    throw err; // Re-throw if caller should handle it
  }
}

---

The waterfall pattern is fundamental for handling dependent async operations. While async.waterfall from the async library was essential in the callback era, modern JavaScript’s async/await provides the cleanest implementation. Understand when sequential execution is necessary versus when you can parallelize, and you’ll write more efficient async code.