Anaconda Cloud is Very, Very Slow!

Another Attachment:

Second attachment:

Third Attachment:

Fourth Attachment:

Hi there – if you look at the “CPU usage” note at the top right of the screen you’ll see that you’ve used about 4,000 CPU-seconds, which is more than the 2,000/day limit for your account. That will reset within 24 hours and you’ll have another 2,000 to use, but in the meantime your code will run more slowly. It looks like we need to highlight that better; perhaps the “CPU usage” thing at the top should turn red to highlight issues like that, or something along those lines.

Once the daily CPU usage limit has reset, you’ll see the numbers at the top change to 0/2,000. Could you see if your notebook goes back to running at a normal speed after that?

Dear Mr. Thomas,

The CPU usage is high because of my repeated trials.

Should I upgrade to the next level?
Will that give me enough time on your cloud?

–N. Srinivasan

You don’t need to upgrade; at 21:28:03 UTC every day you’ll get another 2,000 seconds (the number at the top right will be updated to reflect that), and things will speed up again until you use up all of your CPU allowance for the day.

Giles,
That is 30 minutes per day.
In development, as a professor, to develop a workable program, it requires much more than 30 minutes per day.

What should I do.?
If I upgrade, will I get more CPU time?

Best Regards,
N. Srinivasan

Hi there,

It’s not 2,000 seconds elapsed time – it’s 2,000 seconds of running code that uses 100% of a CPU, excluding time it spends while the CPU is idle. For example, code that loads a lot of data from disk might take several seconds to run, but most of the time it would be waiting for the disk to provide data, so it would likely only take microseconds of CPU time over those several seconds.

Likewise, if you’re developing code, the time while you’re typing stuff in doesn’t count. It’s only the time that your code is running and using CPU that would count. The code in your screenshots would only use a tiny amount of CPU time, but – to take an extreme example on the other hand – this rather ugly busy-wait code would use up CPU time at one second per second of real time for ten seconds:

import time

start = time.time()
while time.time() - start < 10:
    pass

On a typical day of coding all of the time I normally use a few hundred CPU seconds by that count – though I must admit that my own coding tends to be more systems-related rather than heavy number-crunching, so perhaps our experiences differ on that point.

Just to answer your question about upgrading – right now all plans have 2,000 CPU-seconds per day; the advantages of the more expensive plans are in other areas. We’re looking into options for providing more CPU-seconds in the future (and I’m sure your comments here will be taken into account when planning that!)

Giles,

Thanks for your input.
I am actually headed towards neural networks-- forward pass and backward pass.
My PC does not have enough memory.
That’s one of the reasons I am considering running it in the cloud.

I do not want to move to AWS or Azure.
I do hope Anaconda will consider a far more liberal policy on CPU time.

Thanks.

Thanks for the feedback! Right now the notebook service is one-size-fits-all, but that’s just the first release and we’re working on expanding the offering so that there are more options available. How much do you think you would need in terms of both memory and CPU for your projects? And are there any other features that would be necessary – for example, do you think GPU support might be of interest? The more feedback we get, the more we can be sure we’re building what people want :slight_smile:

Thanks for your reply!
Increasingly, Python and its applications are moving towards Neural Networks. This implies training the network through Forward Pass and Backward Pass multiple times.
Libraries such as PyTorch recommend the use of the GPU to speed up the runs.
As far as memory goes, 256 GB per application (per person) should suffice.

So far I have been running my applications on my laptop with 16 GB of memory. I have hit the ceiling it seems. So, I just ordered a workstation with 128GB of RAM and 2TM of SSD, with 8GB NVDIA Graphics card. It has not arrived yet. I certainly will let you know the performance!

Giles,
I am encountering a problem in installing scikit-learn. the details are given below.
It seems that the problem is within Anaconda. The error message is too cryptic for me to understand. I will be much obliged if you can fix this problem.

(base) PS C:\Users\test\Documents\PYTORCH> conda activate pytorch

(pytorch) PS C:\Users\test\Documents\PYTORCH> python

Python 3.8.13 (default, Oct 19 2022, 22:38:03) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32

Type “help”, “copyright”, “credits” or “license” for more information.

(pytorch) PS C:\Users\test\Documents\PYTORCH> conda install scikit-learn

Collecting package metadata (current_repodata.json): done

Solving environment: done

Package Plan

environment location: C:\Users\test.conda\envs\pytorch

added / updated specs:

  • scikit-learn

The following NEW packages will be INSTALLED:

fftw pkgs/main/win-64::fftw-3.3.9-h2bbff1b_1 None

icc_rt pkgs/main/win-64::icc_rt-2022.1.0-h6049295_2 None

joblib pkgs/main/win-64::joblib-1.1.1-py38haa95532_0 None

scikit-learn pkgs/main/win-64::scikit-learn-1.1.2-py38hd77b12b_0 None

scipy pkgs/main/win-64::scipy-1.9.1-py38he11b74f_0 None

threadpoolctl pkgs/main/noarch::threadpoolctl-2.2.0-pyh0d69192_0 None

Proceed ([y]/n)?y

Preparing transaction: done

Verifying transaction: done

Executing transaction: -

Windows 64-bit packages of scikit-learn can be accelerated using scikit-learn-intelex.

More details are available here: https://intel.github.io/scikit-learn-intelex

For example:

$ conda install scikit-learn-intelex

$ python -m sklearnex my_application.py

done

ERROR conda.core.link:_execute(733): An error occurred while installing package ‘defaults::scikit-learn-1.1.2-py38hd77b12b_0’.

Rolling back transaction: done

LinkError: post-link script failed for package defaults::scikit-learn-1.1.2-py38hd77b12b_0

location of failed script: C:\Users\test.conda\envs\pytorch\Scripts.scikit-learn-post-link.bat

==> script messages <==

Windows 64-bit packages of scikit-learn can be accelerated using scikit-learn-intelex.

More details are available here: https://intel.github.io/scikit-learn-intelex

For example:

$ conda install scikit-learn-intelex

$ python -m sklearnex my_application.py

==> script output <==

stdout: subdir: win-64

stderr: ‘chcp’ is not recognized as an internal or external command,

operable program or batch file.

‘chcp’ is not recognized as an internal or external command,

operable program or batch file.

‘chcp’ is not recognized as an internal or external command,

operable program or batch file.

return code: 1

()

(pytorch) PS C:\Users\test\Documents\PYTORCH>

Hi there! Regarding your second post, I was going to suggest that you posted in the other part of the forums specific to local installations of Conda, but I see you’ve already done that. I’ll post regarding your previous comment about machine specs next.

So, on to the specs required for neural nets:

So, I just ordered a workstation with 128GB of RAM and 2TM of SSD, with 8GB NVDIA Graphics card. It has not arrived yet. I certainly will let you know the performance!

Wow, that’s a lovely machine – I’m quite jealous :slight_smile:

I’d be very interested in hearing how it performs. In particular, though, I’d love to know how much of the time you find yourself using it at 100% capacity – say, training your NNs – and how much time per day you use it at a lower capacity (editing code and so on). We’re still planning out what the future enhancements to our notebook service will be, and this would be really valuable data if you’re willing to share it.

The reason for that is that cloud computing services are cheaper than using your own machine because they can try to run at 100% capacity all of the time – that is, while one person is doing a training run that’s using the GPU, another might be editing stuff and not using much capacity, and another person might have finished for the day and not be using the system at all. Sharing the resources brings down the price for everyone.

So if you’re willing to tell us how much you find yourself using your machine, that gives us an indication of what kind of usage we might expect from people with use cases like yours if and when we add that kind of plan to our system, which tells us what capacity each such person using the system might need, and ultimately helps us get the answer to the important question of the price!

Diles,

Thanks for your response!!

“The reason for that is that cloud computing services are cheaper than using your own machine because they can try to run at 100% capacity all of the time – that is, while one person is doing a training run that’s using the GPU, another might be editing stuff and not using much capacity, and another person might have finished for the day and not be using the system at all. Sharing the resources brings down the price for everyone.”

Yes, absolutely true!

That’s why I signed for anaconda.cloud.

So if you’re willing to tell us how much you find yourself using your machine, that gives us an indication of what kind of usage we might expect from people with use cases like yours if and when we add that kind of plan to our system, which tells us what capacity each such person using the system might need, and ultimately helps us get the answer to the important question of the price!

Yes, I will be happy to share my experience after the new machine arrives.

It seems that the “Starter” package can charge $20 per month, $240 per year, 10,000 Seconds of CPU use per day, 20 GB of storage, and if the CPU use time exceeds that quota, charge an extra 1 cent per second of CPU use time.

This is my suggested first tier. You can take it from there.

At this point it is hard to say how much a neural network will demand the CPU time. Most of the CPU time, in my experience, is consumed during the training phase, and the number of features.

Brilliant, many thanks! I’ll make sure that your feedback gets fed through into the planning for the next stages for the notebook service. I can’t promise that we’ll go in that direction, of course, but having real users’ thoughts and ideas to guide us is exactly what we need.

(And if other people who are using the notebook service are reading this thread, please do post with your own thoughts too – they’d be much appreciated.)

i m facing the same issue how to resolve CPU usage shows red and notebook is not running

Hi Parirauf15 -
You may have exceeded your CPU usage limit for the day. Our notebook instances have a limit for the maximum number of seconds fully utilizing the CPU. Once an instance hits that limit, it is not shut down, but instead given lower CPU priority and a limit to the amount of compute resources available. This limit is reset every day, so full compute access will be restored the next day.

To see current progress towards your daily quota, reference the widget in the upper right of the interface that shows current CPU usage vs. the daily limit.

To better manage your CPU usage, regularly check the Running Terminals and Kernels widget in the left sidebar to kill unnecessary kernels when you no longer need them.

Here is a link to the Anaconda Notebooks FAQ as well: Anaconda Notebooks FAQ — Anaconda Documentation

Hope this helps.
Priya