Why does this function use 100% CPU?

private static string GetProxy()
{
    var rnd = new Random();
    if (Settings.Globals.UsedProxies.Count >= 100)
    {
        Settings.Globals.UsedProxies.Clear();
    }

Start:
    var inx = rnd.Next(0, Settings.Globals.Proxies.Count);
    var theProx = Settings.Globals.Proxies[inx];
    foreach (var item in Settings.Globals.UsedProxies)
    {
        if (item == theProx)
            goto Start;
    }
    Settings.Globals.UsedProxies.Add(theProx);
    return theP开发者_运维知识库rox;
}

I call this code from a pool of 5 threads on random intervals from 10 to 30 seconds. This uses 100% CPU and lags the system very bad. If I comment out my call to GetProxy the app uses only 7% CPU. Any ideas?

The idea is I have a list with 1000 proxies. Once a proxie is used I want to add it to the usedproxies list and never use a proxy thats already been used.

Your funny goto loop is guaranteed to run forever.
Your code picks a random item from the list, loops until it finds that item, and starts over.

Once all of the proxies are used, your code will loop forever, because it can't find any more proxies to add.

Also, List<T> is not thread-safe, so your code is likely to fail in unpredictable ways.

This is strictly speaking not an answer to the OPs question (why does this function take 100% CPU), however OPs has problems with race condition which may make lists behave erratically. So I thought I could demonstrate one way to deal with that

As far as I understand the code allocates a random proxy string from a list of proxies. The code checks if this is already in free, if it's not it tries to pick another proxy string.

One of the problems with the code is that it is stated that this code is called concurrently but the code isn't safe to access concurrently.

One way to deal with it is introducing a ProxyPool class that handles concurrent access safely.

Below is some code which might be of use as a starting point on how to build a ProxyPool class:

namespace SO_ProxyPool
{
    using System;
    using System.Collections.Generic;
    using System.Diagnostics;
    using System.Linq;
    using System.Threading.Tasks;

    sealed class ProxyPool
    {
        readonly object m_lock = new object ();
        readonly Random m_random = new Random ();
        readonly HashSet<string> m_usedProxies = new HashSet<string>();
        readonly HashSet<string> m_freeProxies = new HashSet<string>();
        volatile int m_minSize;

        public ProxyPool (IEnumerable<string> availableProxies)
        {
            m_freeProxies = new HashSet<string> (availableProxies);
            m_minSize = m_freeProxies.Count;
        }

        /// <summary>
        /// Reserves a proxy, returns null if no proxy is available
        /// </summary>
        /// <returns>The reserver proxy or null if no proxy is available</returns>
        public string ReserveProxy ()
        {
            lock (m_lock)
            {
                if (m_freeProxies.Count == 0)
                {
                    return null;
                }

                var index = m_random.Next (0, m_freeProxies.Count);

                var proxy = m_freeProxies.ElementAt (index);

                var removeSuccessful = m_freeProxies.Remove (proxy);
                var addSuccessful = m_usedProxies.Add (proxy);
                Debug.Assert (removeSuccessful);
                Debug.Assert (addSuccessful);

                m_minSize = Math.Min (m_minSize, m_freeProxies.Count);

                return proxy;
            }
        }

        /// <summary>
        /// Returns the minimum size of the pool so far
        /// </summary>
        public int MinSize
        {
            get
            {
                return m_minSize;
            }
        }

        /// <summary>
        /// Frees a reserved proxy
        /// </summary>
        /// <param name="proxy">The proxy to free</param>
        public void FreeProxy (string proxy)
        {
            if (proxy == null)
            {
                return;
            }

            lock (m_lock)
            {
                var removeSuccessful = m_usedProxies.Remove (proxy);
                if (removeSuccessful)
                {
                    var addSuccessful = m_freeProxies.Add (proxy);
                    Debug.Assert (addSuccessful);
                }

            }
        }
    }

    class Program
    {
        static readonly ProxyPool s_proxyPool = new ProxyPool (
            new[] { "0", "1", "2", "3", "4", "5", "6", "7", "8", "9", }
            );

        static string GetProxy ()
        {
            return s_proxyPool.ReserveProxy ();
        }

        static void FreeProxy (string proxy)
        {
            s_proxyPool.FreeProxy (proxy);
        }

        static void SimplisticTestCase ()
        {
            var proxy = GetProxy ();
            // Do something relevant...
            if (proxy != null)
            {
                FreeProxy (proxy);
            }
        }

        static void Main (string[] args)
        {
            var then = DateTime.Now;

            const int count = 10000000;
            Parallel.For (0, count, idx => SimplisticTestCase ());

            var diff = DateTime.Now - then;

            Console.WriteLine (
                "#{0} executions took {1:0.00}secs, pool min size {2}", 
                count,
                diff.TotalSeconds,
                s_proxyPool.MinSize
                );
        }
    }
}

To answer the actual question, it uses 100% of the CPU (on a single core machine I assume) because everything is small enough to fit in memory and we're just looping through and doing some checks. This is very CPU intensive.

To create a list of Unused proxies you can do the following:

HashSet unused = new HashSet(Settings.Globals.Proxies);
List unused = all.ExceptWith(Settings.Globals.UsedProxies);
unused.ExceptWith(Settings.Globals.UsedProxies);

then select a random proxy from the unused set using the unused.Count property and unused.GetEnumerator().

Try this (assuming all proxies in UsedProxies can be found in Proxies):

List<string> unusedProxies = new List<string>(Settings.Globals.Proxies);
foreach (string proxy in Settings.Globals.UsedProxies)
{
    unusedProxies.Remove(proxy);
}

int inx = rnd.Next(0, unusedProxies.Count);
string proxy = unusedProxies[inx];
Settings.Globals.UsedProxies.Add(proxy);
return proxy;

This should be faster than your version, since all the unused proxies are in their own separate list. Then you can use rnd.Next to get a random proxy, and that proxy is guaranteed to be unused.