Metadata-Version: 1.1
Name: ck
Version: 1.6.6
Summary: Collective Knowledge - lightweight knowledge manager to organize, cross-link, share and reuse artifacts
Home-page: https://github.com/ctuning/ck/wiki
Author: Grigori Fursin and non-profit cTuning foundation
Author-email: Grigori.Fursin@cTuning.org
License: BSD 3-clause
Description: Collective Knowledge Infrastructure - enabling open,
        collaborative and reproducible experimentation, 
        knowledge sharing and predictive analytics.
        
        License
        =======
        * Permissive 3-clause BSD license 
        (see LICENSE.txt file for more details).
        
        Requirements
        ============
        * has been validated on Linux, Windows, Android (partially), Mac OS X
        * only python >= 2.6 (3.0+ is natively supported)
        
        Optional dependencies
        =====================
        * Python package 'Tkinter' or 'tkinter' or 'pyperclip' 
          if copy to clipboard functionality is required
        * Python package 'matplotlib' to plot graphs
          (during autotuning or performance/energy/accuracy modeling)
        * Python package 'psutil' to properly terminate running process 
          on Windows after timeout
        
        Introduction
        ============
        
        We have developed Collective Knowledge Framework and Repository (CK)
        primarily to solve numerous problems we faced during past 20 years
        of our research on building self-optimizing computer systems
        (combining performance autotuning, machine learning and run-time adaptation):
        
        * numerous and ever changing hardware
        * black-box compilers
        * ad-hoc tools with evolving and incompatible interfaces
        * multiple heterogeneous and possibly proprietary data formats
        * raising number of design and optimization choices
        * raising amount of experimental data to process
        * ad-hoc, outdated and non-representative benchmarks with limited data sets
        * lack of unified and reproducible mechanisms for exchange of experimental results
        * problems accessing proprietary software (compilers, benchmarks)
          and hardware for artifact evaluation
        
        Eventually, we created CK as an open-source and light-weight SDK
        (~200K python code) to help our colleagues preserve, organize,
        describe, cross-link and share their code and data (benchmarks,
        data sets, scripts, libraries, tools, experimental results,
        models, tables, graphs, articles, etc) as reusable Python-based
        components with a very simple API (just one function with
        JSON in and JSON out) and JSON-based schema-free meta description, 
        as shown in the following examples:
        * https://github.com/ctuning/ctuning-programs 
        * https://github.com/ctuning/ctuning-datasets-min
        * https://github.com/ctuning/ck-analytics
        * https://github.com/ctuning/ck-autotuning
        
        CK also helps abstract access to ever-changing (and possibly
        proprietary) tools and hardware via wrappers with the same JSON
        API while protecting experimental setups from low-level
        interface/data changes (wrappers handle such changes as well
        as setting up environment for multiple versions of a given 
        and possibly pre-installed tool or library). 
        
        All components has DOI-style UID and can be transparently indexed 
        and searched via third-party Hadoop-based ElasticSearch, 
        shared via GIT, and connected together into experimental pipelines 
        (workflows) just as LEGO(TM) to quickly prototype various research ideas, 
        crowdsource experiments, reproduce and improve past techniques.
        
        Furthermore, CK can simplify connection of unified experiments 
        to powerful predictive analytics tools such as scikit-learn 
        and R (statistical analysis, data mining,  machine learning)
        thus helping non-specialists perform statistical analysis of results,
        automate  exploration of large, multi-dimensional design 
        and optimization choices, speed up decision making, and
        easily share results with colleagues via interactive graphs 
        and articles, as demonstrated in our latest CK-powered interactive publications:
        * http://cknowledge.org/repo/web.php?wcid=29db2248aba45e59:cd11e3a188574d80
        * http://cknowledge.org/repo/web.php?wcid=29db2248aba45e59:6f40bc99c4f7df58
        
        Unlike existing centralized web-based services 
        that force users to upload all their code and data before
        being processed, CK allows users organize their local artifacts 
        and always keep track of them on their own machine 
        while sharing them only if needed as P2P or via existing
        private and public repositories.
        
        CK can also complement existing technology such as Docker 
        and VM hardwired images by decomposing them into unified 
        and reusable components with a simple, extensible and unified JSON API.
        This is particularly useful for computer systems' R&D where 
        software and even hardware is changing every day and becomes 
        outdated very quickly. Hence, researchers would often like to 
        be able  to actually rebuild, customize and rerun experimental 
        setups using their own native tools (latest or different operating 
        systems, compilers and libraries) rather than using VM images - 
        CK was also developed to solve this issue. For example,
        various research projects already use CK to share their
        research artifacts and scenarios:
        $ http://github.com/ctuning/reproduce-carp-project
        $ http://github.com/ctuning/reproduce-pamela-project
        $ http://github.com/ctuning/reproduce-clsmith
        $ http://github.com/ctuning/reproduce-ck-paper 
        
        CK can be invoked in a unified way via command line or JSON web service.
        It can also be directly invoked from other programs and tools
        written in practically any language including C, C++, Fortran,
        PHP and Java using OpenME event-based plugin framework (we worked
        with the community to add similar plugin interface to GCC 4.6+
        and plan to add it to LLVM).
        
        Eventually, unified mechanisms of artifact and knowledge exchange
        in the CK allows the community to gradually validate and improve
        shared techniques and data sets thus enabling truly open,
        collaborative, interdisciplinary and reproducible research
        similar to physics and other natural sciences. To some extent,
        it can help computer engineers and researchers become data scientists 
        and focus on innovation while liberating them from ad-hoc,
        repetitive, boring and time consuming tasks.
        It should also help solve some of big data issues we faced
        since 1993 by actively using and sharing predictive models 
        instead of large amount of data.
        
        You may check out our motivation as well as real usage scenarios,
         CK-powered interactive papers and websites (some are based on previous 
        version aka Collective Mind) here:
        
        * [Interactive article 1] http://cknowledge.org/repo/web.php?wcid=29db2248aba45e59:6f40bc99c4f7df58
        * [Interactive article 2] http://cknowledge.org/repo/web.php?wcid=29db2248aba45e59:cd11e3a188574d80
        * [All shared artifacts via CK]: http://github.com/ctuning/reproduce-ck-paper
        * [Our open publication model] http://arxiv.org/abs/1406.4020
        * [Live CK repository] http://cknowledge.org/repo
        * [CK-generated interactive CV] http://fursin.net/cv.html
        * [ADAPT workshop with Reddit-based reviewing] http://adapt-workshop.org
        * [Android client for crowd-benchmarking] https://play.google.com/store/apps/details?id=com.collective_mind.node
        
        Further details about CK including getting started guide, 
        real usage scenarios, and developer guide is available here:
        
        * https://github.com/ctuning/ck/wiki
        
        Copyright
        =========
        This software is being developed by the non-profit 
        cTuning foundation and its volunteers.
        
        (C)opyright 2014-2015 Grigori Fursin, 
        cTuning foundation and contributors
        
        http://cTuning.org
        
        See COPYRIGHT.txt for more details.
        
        Authors
        =======
        * Grigori Fursin, http://fursin.net
        * Anton Lokhmotov, https://www.hipeac.net/~anton
        
        Installation
        ============
        We made a special effort to make CK installation 
        as simple as possible - it requires only standard 
        Python version 2.6 or above (3.x is supported).
        
        CK has been successfully tested by volunteers on various 
        platforms including Ubuntu, OpenSUSE, CentOS, MacOS X, 
        Android (partial support through web services and OpenME) 
        and most of the recent Windows versions. 
        
        Note: since user packages may require extra functionality
        such as scipy and scikit-learn, we have been 
        successfully using Anaconda python distribution
        which includes all scientific packages required 
        for predictive analytics in CK.
        
        You can install CK either via GIT (latest self-updatable 
        development version) or via PIP/Conda.
        
        ### PIP installation
         $ pip install ck
        
        ### GIT installation
        
         $ git clone https://github.com/ctuning/ck.git ck
         $ cd ck
         $ (sudo) python setup.py install
        
        Now, you can simply add CK bin directory to your OS PATH 
        environment variable and you are ready to go. 
        
        Usage
        =====
        
        You can check that CK works via CMD:
        
         $ ck
        
        You should see information about ck options.
        
        If you have IPython installed, you can then check CK installation
        as following:
        
         $ ipython
        
         $ import ck.kernel as ck
        
         $ ck.test()
        
         $ ck.access('list module')
        
         $ ck.access('load kernel default')
        
        Now, you can use CK to pull various public CK repositories
        (code and data shared as reusable components via GitHub
        or Bitbucket) and run shared experimental workflows
        (for example, to crowd-benchmark and autotune programs,
        build predictive models of performance, analyze representative
        benchmarks and data sets, install missing tools, ...).
        
        For example, compiling and running any shared benchmark 
        on your Linux machine with GCC can be done simply as:
        
         $ ck pull repo:ctuning-programs
        
         $ ck compile program:cbench-automotive-susan --speed
        
         $ ck run program:cbench-automotive-susan
        
        Compiling and running such benchmarks on Windows or Android,
        autotuning or crowdtuning them (performance, energy, accuracy, size, etc),
        adding more data sets, applying machine learning to predict optimizations,
        and any other shared scenario requires a few more steps 
        as described in our "Getting Started Guide":
        
        * https://github.com/ctuning/ck/wiki
        
        Adding new CK modules as containers for own data, as a wrapper
        for some tool or as a workflow is also straightforward. 
        For example, you can add your own module 'hello' with an action 'say'
        and with an entry 'world' as following (you can select all
        default values by just pressing Enter):
        
         $ ck add module:hello
        
         $ ck add_action module --func=say
        
         $ ck add hello:world
        
         $ ck say hello:world
        
        You can now find and customize Python dummy function 'say' 
        in the CK module 'hello' (module.py) via
         
         $ ck find module:hello
        
        You can also find associated CK entry hello:say and modify 
        its meta information (.cm/meta.json) via
        
         $ ck find hello:world
        
        Please, follow CK guide for more details:
        
        * https://github.com/ctuning/ck/wiki
        
        Questions/comments/discussions?
        ===============================
        Please, use our mailing lists:
        * General topics of knowledge preservation, sharing and reuse
          as well as collaborative and reproducible R&D: http://groups.google.com/group/collective-knowledge
        * Software and hardware performance/energy/size/reliability
          (auto/crowd)tuning: http://groups.google.com/group/ctuning-discussions
        
        Publications
        ============
        Concepts has been described in the following publications:
        
        * http://arxiv.org/abs/1506.06256
        * http://hal.inria.fr/hal-01054763
        * http://arxiv.org/abs/1406.4020
        * https://hal.inria.fr/inria-00436029
        
        If you found CK useful and/or interesting, you are welcome
        to reference any of the above publications in your articles
        and reports.
        
        Fun
        ===
        * CK GIT commits as a video: http://cknowledge.org/soft/commits/ck-visualization.mp4
        
        CK-powered projects
        ===================
        
        CK is currently used in the following projects:
        
        * building public repository of benchmarks, data sets, tools, predictive
        models and optimization knowledge in a unified format with the help
        of the computer engineering community
        (http://cknowledge.org/repo , http://c-mind.org/repo)
        
        * implementing universal multi-dimensional, multi-objective,
        plugin-based autotuning combined with crowdsourcing, predictive
        analytics and run-time adaptation (to enable self-optimizing
        computer systems). We support OpenCL, CUDA, OpenMP, MPI, compiler
        and any other tuning for performance, energy, size, reliability,
        cost and any other metrics across small kernels/codelets and
        large applications (http://cknowledge.org/repo/web.php?wcid=29db2248aba45e59:cd11e3a188574d80).
        
        * implementing crowd-benchmarking (crowdsourcing workload characterization
        and compiler heuristic construction across numerous architectures using shared
        computational resources such as mobile phones, tablets, cloud services, etc.
        (https://play.google.com/store/apps/details?id=com.collective_mind.node ,
        http://cknowledge.org/repo/web.php?wcid=29db2248aba45e59:cd11e3a188574d80)
        
        * supporting artifact evaluation initiatives for major conferences
        and journals where all artifacts are shared as reusable components (and not
        just as black box virtual machine images) along with publications
        (http://ctuning.org/event/ae-ppopp2016 , http://arxiv.org/abs/1406.4020 ,
        http://adapt-workshop.org)
        
        * enabling open, collaborative and reproducible research and experimentation
        with interactive publications focusing on computer engineering
        ( http://cTuning.org/reproducibility-wiki )
        
        * enabling interactive and reproducible articles for Digital Libraries
        (http://cknowledge.org/repo/web.php?wcid=29db2248aba45e59:cd11e3a188574d80 ,
        http://cknowledge.org/repo/web.php?wcid=29db2248aba45e59:6f40bc99c4f7df58)
        
        * serving as a personal knowledge manager to organize, interconnect
        and preserve all personal coda and data via simple JSON
        meta with UIDs and semantic tags.
        
        You are welcome to add your own CK-powered project here!
        
        Acknowledgments
        ===============
        
        CK development is coordinated by the non-profit cTuning foundation
        (cTuning.org). We would like to thank the EU TETRACOM 609491 project
        (www.tetracom.eu) for initial funding and dividiti (www.dividiti.com)
        for continuing support. We are also extremely grateful to all volunteers
        for their valuable feedback and contributions.
        
Platform: UNKNOWN
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Console
Classifier: Environment :: Plugins
Classifier: Environment :: Web Environment
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: End Users/Desktop
Classifier: Intended Audience :: Information Technology
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: BSD License
Classifier: Operating System :: OS Independent
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Software Development
Classifier: Topic :: System
Classifier: Topic :: System :: Benchmark
Classifier: Topic :: Education
Classifier: Topic :: Home Automation
Classifier: Topic :: Adaptive Technologies
Classifier: Topic :: Database
Classifier: Topic :: Utilities
