memory usage

[Alexandre Riveira]

I usually use XEpollThreadPool however persebi a large memory usage for
sites that have a lot of content and access reaching over 1 GB. Changed
to ThreadPool and memory fell not from 4x 256MB. However
XEpollThreadPool is faster than ThreadPool. Accepted suggestions for
configuration. I tested also with CoolioThreadPool speed was good but
the memory consumption and large too.


Tanks


Alexandre Riveira
_______________________________________________
Rainbows! mailing list - rainbows-talk-***@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying

[Eric Wong]

Post by Alexandre Riveira
I usually use XEpollThreadPool however persebi a large memory usage
for sites that have a lot of content and access reaching over 1 GB.
Changed to ThreadPool and memory fell not from 4x 256MB. However
XEpollThreadPool is faster than ThreadPool. Accepted suggestions for
configuration. I tested also with CoolioThreadPool speed was good
but the memory consumption and large too.

Wait, ThreadPool uses significantly less than XEpollThreadPool?

Since this is likely glibc/eglibc, try setting MALLOC_ARENA_MAX=1 and
MALLOC_ARENA_TEST=1 in the env before starting Rainbows! AFAIK, these
are mainly documented by Red Hat, but in all recent-ish versions of
glibc/eglibc support it.

Then increase the values if you see performance gains (unlikely unless
your app releases the GVL a lot or don't have one (rbx)).
_______________________________________________
Rainbows! mailing list - rainbows-talk-***@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying

[Alexandre Riveira]

Yes, XEpollThreadPool 4 times more memory than ThreadPool will do tests
as you indicated and verifying systems that have other glibc

Tanks

Alexandre Riveira

Post by Eric Wong

Post by Alexandre Riveira
I usually use XEpollThreadPool however persebi a large memory usage
for sites that have a lot of content and access reaching over 1 GB.
Changed to ThreadPool and memory fell not from 4x 256MB. However
XEpollThreadPool is faster than ThreadPool. Accepted suggestions for
configuration. I tested also with CoolioThreadPool speed was good
but the memory consumption and large too.

Wait, ThreadPool uses significantly less than XEpollThreadPool?
Since this is likely glibc/eglibc, try setting MALLOC_ARENA_MAX=1 and
MALLOC_ARENA_TEST=1 in the env before starting Rainbows! AFAIK, these
are mainly documented by Red Hat, but in all recent-ish versions of
glibc/eglibc support it.
Then increase the values if you see performance gains (unlikely unless
your app releases the GVL a lot or don't have one (rbx)).
_______________________________________________
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying

_______________________________________________
Rainbows! mailing list - rainbows-talk-***@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying

[Alexandre Riveira]

Hello Eric,

I discovered that when you restart the server (linux 2.3.50, glibc 2.15)
memory stopped growing indiscriminately.

2 questions:

1) The POOL_SIZE XEpollThreadPool series of the number of connections
accepted epoll so'd have one for each thread
2) I saw the gem 'mall' and she seems to adjust as you indicted by glibc
environment variables are correct? If yes in config / initializers added
a file.rb with the following:
Mall.opt (Mall :: ARENA_MAX, 1)
Mall.opt (Mall :: ARENA_TEST, 1)


Tanks,

Alexandre Riveira

Post by Eric Wong

Post by Alexandre Riveira
I usually use XEpollThreadPool however persebi a large memory usage
for sites that have a lot of content and access reaching over 1 GB.
Changed to ThreadPool and memory fell not from 4x 256MB. However
XEpollThreadPool is faster than ThreadPool. Accepted suggestions for
configuration. I tested also with CoolioThreadPool speed was good
but the memory consumption and large too.

Wait, ThreadPool uses significantly less than XEpollThreadPool?
Since this is likely glibc/eglibc, try setting MALLOC_ARENA_MAX=1 and
MALLOC_ARENA_TEST=1 in the env before starting Rainbows! AFAIK, these
are mainly documented by Red Hat, but in all recent-ish versions of
glibc/eglibc support it.
Then increase the values if you see performance gains (unlikely unless
your app releases the GVL a lot or don't have one (rbx)).
_______________________________________________
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying

_______________________________________________
Rainbows! mailing list - rainbows-talk-***@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying

[Eric Wong]

Post by Alexandre Riveira
Hello Eric,
I discovered that when you restart the server (linux 2.3.50, glibc
2.15) memory stopped growing indiscriminately.

Linux 2.3.50? Huh? Assuming that's 3.2.50. Anyways, user-visible
memory usage is only dependent on glibc version.

Post by Alexandre Riveira
1) The POOL_SIZE XEpollThreadPool series of the number of
connections accepted epoll so'd have one for each thread

No, pool_size for XEpollThreadPool is the number of worker threads
capable of dispatching the application. It can hold many more clients.

Post by Alexandre Riveira
2) I saw the gem 'mall' and she seems to adjust as you indicted by
glibc environment variables are correct? If yes in config /
Mall.opt (Mall :: ARENA_MAX, 1)
Mall.opt (Mall :: ARENA_TEST, 1)

Yes, mall works, too, and you can tune a running app with some of the
knobs. However, for arena settings, I think they must be set early on
(before many threads are spawned) to be effective. So environment
variables (set before starting Ruby) are probably the best bet, since
Ruby (MRI) spawns a background timer thread right away.
_______________________________________________
Rainbows! mailing list - rainbows-talk-***@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying

[Alexandre Riveira]

Hello Eric,
My linux is 3.2.50, sorry my error, environment variables work fine now.

Tanks for your help

Alexandre Riveira

Post by Eric Wong

Post by Alexandre Riveira
Hello Eric,
I discovered that when you restart the server (linux 2.3.50, glibc
2.15) memory stopped growing indiscriminately.

Linux 2.3.50? Huh? Assuming that's 3.2.50. Anyways, user-visible
memory usage is only dependent on glibc version.

Post by Alexandre Riveira
1) The POOL_SIZE XEpollThreadPool series of the number of
connections accepted epoll so'd have one for each thread

No, pool_size for XEpollThreadPool is the number of worker threads
capable of dispatching the application. It can hold many more clients.

Post by Alexandre Riveira
2) I saw the gem 'mall' and she seems to adjust as you indicted by
glibc environment variables are correct? If yes in config /
Mall.opt (Mall :: ARENA_MAX, 1)
Mall.opt (Mall :: ARENA_TEST, 1)

Yes, mall works, too, and you can tune a running app with some of the
knobs. However, for arena settings, I think they must be set early on
(before many threads are spawned) to be effective. So environment
variables (set before starting Ruby) are probably the best bet, since
Ruby (MRI) spawns a background timer thread right away.
_______________________________________________
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying

_______________________________________________
Rainbows! mailing list - rainbows-talk-***@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying