spin_lock_irqsave vs spin_lock_irq

On an SMP machine we must use spin_lock_irqsave and not spin_lock_irq from interrupt context.

Why would we want to save the flags (which contain the IF)?

Is there another interrupt routine t开发者_开发技巧hat could interrupt us?

spin_lock_irqsave is basically used to save the interrupt state before taking the spin lock, this is because spin lock disables the interrupt, when the lock is taken in interrupt context, and re-enables it when while unlocking. The interrupt state is saved so that it should reinstate the interrupts again.

Example:

1. Lets say interrupt x was disabled before spin lock was acquired
2. spin_lock_irq will disable the interrupt x and take the the lock
3. spin_unlock_irq will enable the interrupt x.

So in the 3rd step above after releasing the lock we will have interrupt x enabled which was earlier disabled before the lock was acquired.

So only when you are sure that interrupts are not disabled only then you should spin_lock_irq otherwise you should always use spin_lock_irqsave.

If interrupts are already disabled before your code starts locking, when you call spin_unlock_irq you will forcibly re-enable interrupts in a potentially unwanted manner. If instead you also save the current interrupt enable state in flags through spin_lock_irqsave, attempting to re-enable interrupts with the same flags after releasing the lock, the function will just restore the previous state (thus not necessarily enabling interrupts).

Example with spin_lock_irqsave:

spinlock_t mLock = SPIN_LOCK_UNLOCK;
unsigned long flags;

spin_lock_irqsave(&mLock, flags); // Save the state of interrupt enable in flags and then disable interrupts
// Critical section
spin_unlock_irqrestore(&mLock, flags); // Return to the previous state saved in flags

Example with spin_lock_irq( without irqsave ):

spinlock_t mLock = SPIN_LOCK_UNLOCK;
unsigned long flags;

spin_lock_irq(&mLock); // Does not know if interrupts are already disabled
// Critical section
spin_unlock_irq(&mLock); // Could result in an unwanted interrupt re-enable...

The need for spin_lock_irqsave besides spin_lock_irq is quite similar to the reason local_irq_save(flags) is needed besides local_irq_disable. Here is a good explanation of this requirement taken from Linux Kernel Development Second Edition by Robert Love.

The local_irq_disable() routine is dangerous if interrupts were already disabled prior to its invocation. The corresponding call to local_irq_enable() unconditionally enables interrupts, despite the fact that they were off to begin with. Instead, a mechanism is needed to restore interrupts to a previous state. This is a common concern because a given code path in the kernel can be reached both with and without interrupts enabled, depending on the call chain. For example, imagine the previous code snippet is part of a larger function. Imagine that this function is called by two other functions, one which disables interrupts and one which does not. Because it is becoming harder as the kernel grows in size and complexity to know all the code paths leading up to a function, it is much safer to save the state of the interrupt system before disabling it. Then, when you are ready to reenable interrupts, you simply restore them to their original state:

unsigned long flags;

local_irq_save(flags);    /* interrupts are now disabled */ /* ... */
local_irq_restore(flags); /* interrupts are restored to their previous
state */

Note that these methods are implemented at least in part as macros, so the flags parameter (which must be defined as an unsigned long) is seemingly passed by value. This parameter contains architecture-specific data containing the state of the interrupt systems. Because at least one supported architecture incorporates stack information into the value (ahem, SPARC), flags cannot be passed to another function (specifically, it must remain on the same stack frame). For this reason, the call to save and the call to restore interrupts must occur in the same function.

All the previous functions can be called from both interrupt and process context.

Reading Why kernel code/thread executing in interrupt context cannot sleep? which links to Robert Loves article, I read this :

some interrupt handlers (known in Linux as fast interrupt handlers) run with all interrupts on the local processor disabled. This is done to ensure that the interrupt handler runs without interruption, as quickly as possible. More so, all interrupt handlers run with their current interrupt line disabled on all processors. This ensures that two interrupt handlers for the same interrupt line do not run concurrently. It also prevents device driver writers from having to handle recursive interrupts, which complicate programming.

Below is part of code in linux kernel 4.15.18, which shows that spiin_lock_irq() will call __raw_spin_lock_irq(). However, it will not save any flags as you can see below part of the code but disable the interrupt.

  static inline void __raw_spin_lock_irq(raw_spinlock_t *lock)
    {
        local_irq_disable();
        preempt_disable();
        spin_acquire(&lock->dep_map, 0, 0, _RET_IP_);
        LOCK_CONTENDED(lock, do_raw_spin_trylock, do_raw_spin_lock);
    }

Below code shows spin_lock_irqsave() which saves the current stage of flag and then preempt disable.

static inline unsigned long __raw_spin_lock_irqsave(raw_spinlock_t *lock)
{
    unsigned long flags;

    local_irq_save(flags);
    preempt_disable();
    spin_acquire(&lock->dep_map, 0, 0, _RET_IP_);
    /*
     * On lockdep we dont want the hand-coded irq-enable of
     * do_raw_spin_lock_flags() code, because lockdep assumes
     * that interrupts are not re-enabled during lock-acquire:
     */
#ifdef CONFIG_LOCKDEP
    LOCK_CONTENDED(lock, do_raw_spin_trylock, do_raw_spin_lock);
#else
    do_raw_spin_lock_flags(lock, &flags);
#endif
    return flags;
}

This question starts from the false assertion:

On an SMP machine we must use spin_lock_irqsave and not spin_lock_irq from interrupt context.

Neither of these should be used from interrupt context, on SMP or on UP. That said, spin_lock_irqsave() may be used from interrupt context, as being more universal (it can be used in both interrupt and normal contexts), but you are supposed to use spin_lock() from interrupt context, and spin_lock_irq() or spin_lock_irqsave() from normal context. The use of spin_lock_irq() is almost always the wrong thing to do in interrupt context, being this SMP or UP. It may work because most interrupt handlers run with IRQs locally enabled, but you shouldn't try that.

UPDATE: as some people misread this answer, let me clarify that it only explains what is for and what is not for an interrupt context locking. There is no claim here that spin_lock() should only be used in interrupt context. It can be used in a process context too, for example if there is no need to lock in interrupt context.