Distributed web crawler admin platform for spiders management regardless of languages and frameworks. Distributed crawler management platform that supports any language and framework