SPE Conferences

Over the years with the increasing awareness of system and end user performance a range of conferences have emerged targetting different types of audiences. By far the largest conferences seem to be the ones open to working professionals with interests in various Non Functional aspects of computing including performance. While some of the niche conferences provide platforms for researchers to present cutting edge ideas with regards to desgning, building and delivering systems with focus on specific Non Functional attributes. Other conferences only focus on specific attributes of performance, scalability for given types of systems e.g. HPC (High Performance Computing), etc.

At this page we’ve attempted to put together a list of some of the most familiar conferences that we’ve come across with a primary focus on performance & scalability. The list of organizations/conferences include -

• Computer Measurement Group
• ACM / SPEC ICPE (International Conference on Performance Engineering)
• IEEE ISPASS - International Symposium on Performance Analysis of Systems and Software
• ACM Sigmetrics
• ACM SigSoft - Special Interest Group On Software Engineering
• IFIP WG 7.3
• IEEE Conference on High Performance Computing (HiPC)
• JAX London
• MASCOTS
• Large Scale Testing
• OmniTI Surge
• PNSQC - Pacific Northwest Software Quality Conference
• Qcon
• RAPIDO
• SuperComputing Conference
• STPCon (Software Test Professionals Conference)
• TPCTC (Technology Conference On Performance Evaluation & Benchmarking)
• ValueTools Conference on Performance Evaluation Methodologies and Tools
• Oreilly Velocity Web Performance & Operations
• Workshops On Performance & Reliability (WOPR)
• WebPerfDays

Let’s now take a brief look at each of the organizations and their conferences below -

Computer Measurement Group (CMG)

Extract from the CMG Website, “The Computer Measurement Group is a not-for-profit, worldwide organization of IT professionals committed to sharing information and best practices focused on ensuring the efficiency and scalability of IT service delivery to the enterprise through measurement, quantitative analysis and forecasting. CMG members are primarily concerned with performance evaluation of existing systems to maximize performance (eg. response time, throughput, etc.) and with capacity management where planned enhancements to existing systems or the design of new systems are evaluated to find the necessary resources required to provide adequate performance at a reasonable cost.

CMG holds an international technical conference each year in the United States. The conference consists of a technical program addressing CPE (computer performance evaluation) issues across all popular computing platforms, from open systems to large commercial mainframes; the program contains sessions for amateurs as well as in-depth technical presentations and user experiences. Papers are actively solicited and blind refereed; abstracts are due in early to mid Spring, with actual papers due in in late Spring to early Summer. A late-breaking paper process may be available for technologies recently announced or made generally available closer to the conference. The conference also has an extensive exhibitor program (trade show) with exhibitors displaying their products on the show floor and during exhibitor sessions.”

For more information on the conferences conducted by CMG please Click Here.

ACM / SPEC ICPE (International Conference on Performance Engineering)

Extract from the ICPE Website, “The goal of the International Conference on Performance Engineering (ICPE) is to integrate theory and practice in the field of performance engineering by providing a forum for sharing ideas and experiences between industry and academia. ICPE is established as a joint meeting of the ACM Workshop on Software and Performance (WOSP) and the SPEC International Performance Evaluation Workshop (SIPEW). The conference brings together researchers and industry practitioners to share and present their experiences, discuss challenges, and report state-of-the-art and in-progress research on performance engineering of software and systems, including performance measurement, modeling, benchmark design, and run-time performance management.

Since its inception in 1998, WOSP has brought together software engineers, developers, performance analysts and software/performance modelers who are addressing the challenges of increasing system complexity, rapidly evolving software technologies, short time to market, incomplete documentation, and less-than-adequate methods, models and tools for developing, modeling, and measuring scalable, high-performance software. The focus of WOSP is therefore in the intersection of software and performance, rather than one discipline in isolation.

SIPEW was established by the Standard Performance Evaluation Corporation (SPEC) with the goal to provide a forum for sharing ideas and experiences between industry and academia in the areas of computer benchmarking, performance evaluation, and experimental system analysis. The scope of the workshop includes methodologies, techniques and tools for measurement, load testing, profiling, workload characterization, dependability and efficiency evaluation of computing systems. While the focus is on performance and scalability, other extra-functional system properties such as availability, reliability, cost and energy efficiency are considered as well.”

For more information on ACM / SPEC ICPE (International Conference on Performance Engineering), please Click Here.

IEEE ISPASS - International Symposium on Performance Analysis of Systems and Software

The IEEE International Symposium on Performance Analysis of Systems and Software is sponsored by the IEEE Computer Society’s Technical Committee on Internet, Technical Committee on Computer Architecture, and Technical Committee on Microprogramming and Microarchitecture.

For more information on IEEE ISPASS, please Click Here.

ACM Sigmetrics

SIGMETRICS is the ACM Special Interest Group (SIG) for the computer systems performance evaluation community. SIGMETRICS promotes research in performance analysis techniques as well as the advanced and innovative use of known methods and tools. It sponsors conferences, such as its own annual conference (SIGMETRICS), publishes a newsletter (Performance Evaluation Review), and operates a mailing list linking researchers, students, and practitioners interested in performance evaluation.

Target areas of performance analysis include file and memory systems, database systems, computer networks, operating systems, architecture, distributed systems, fault tolerant systems, and real-time system. In addition, members are interested in developing new performance methodology including mathematical modeling, analysis, instrumentation techniques, model verification and validation, workload characterization, simulation, statistical analysis, stochastic modeling, experimental design, reliability analysis, optimization, and queuing theory.

For more information on the conferences conducted by ACM Sigmetrics please Click Here.

ACM SigSoft - Special Interest Group On Software Engineering

The ACM Special Interest Group on Software Engineering provides a forum for computing professionals from industry, government and academia to examine principles, practices, and new research results in software engineering. SIGSOFT focuses on issues related to all aspects of software development and maintenance. Areas of special interest include: requirements, specification and design, software architecture, validation, verification, debugging, software safety, software processes, software management, measurement, user interfaces, configuration management, software engineering environments, and CASE tools.

ACM SigSoft events and conferences include –

• International Conference On Software Engineering
• International Symposium On The Foundations of Software Engineering
• Automated Software Engineering Conference
• International Symposium on Software Testing & Analysis
• European Software Engineering Conference
• International Symposium on Component Based Software Engineering
• International Symposium on Empirical Software Engineering & Management
• International Conference on Model Driven Languages & Systems
• Workshop on Program Analysis for Software Tools & Engineering
• International Conference On Performance Engineering

For more details on SigSoft and the events they support please Click Here.

IFIP WG 7.3

The IFIP Performance conference aims to bring together researchers interested in understanding and improving the performance of communication systems by means of state-of-the-art quantitative models and solution techniques. The Performance conference has a long-standing tradition and has followed an 18-month cycle for many years. Recently, the interval has been changed to one year.

For more information on the conferences conducted by IFIP WG 7.3 please Click Here.

IEEE Conference on High Performance Computing (HiPC)

HiPC is an international meeting on high performance computing. It serves as a forum to present current work by researchers from around the world as well as highlight activities in Asia in the high performance computing area. The meeting focuses on all aspects of high performance computing systems and their scientific, engineering, and commercial applications.

Topics of interest include but are not limited to:

• High-Performance Computing
• Parallel and Distributed Algorithms/Systems
• Parallel Languages and Programming Environments
• Hybrid Parallel Programming with GPUs and Accelerators
• Load Balancing, Scheduling and Resource Management
• Resilient/Fault-Tolerant Algorithms and Systems
• Scientific/Engineering/Commercial Applications and Workloads
• Emerging Applications such as Biotechnology and Nanotechnology
• Cluster, Cloud, and Grid Computing
• Heterogeneous Computing
• Interconnection Networks and Architectures
• Scalable Servers and Systems
• High Performance/Scalable Storage Systems
• Power-Efficient and Reconfigurable Architectures
• Compiler Technologies for High-Performance Computing
• Software Support and Advanced Micro-architecture Techniques
• Operating Systems for Scalable High-Performance Computing

For more information on the conferences conducted by HiPC please visit Link.

JAX London

JAX London starts with the 3-day conference with 34 worldwide known speakers, 65+ insightful sessions and different tracks on current technologies in the field of Java, Web, Mobile and current data technologies, Performance. JAX is aimed at Academic Students, Practitioners. Known as the place where the most respected thinkers and the brightest minds of the tech community meet, JAX London offers a deep dive for the modern developer and architect aiming to transform open technologies into valuable business solutions. We pride ourselves by always focusing on the big picture: Java core, architecture, and web technologies, as well as expert professional insight into the very latest methodologies and best-practices.

For More Details : Please visit the JAX London 2014 Website at  http://jaxlondon.com/2014/

MASCOTS

The MASCOTS conference is a well-established forum for state-of-the-art research on the measurement, modeling, and performance analysis of computer systems and networks. The 22nd edition of this conference will take place September 9-11, 2014 in Paris, France. The conference will bring together academics and industry practitioners to present and discuss their latest research results. The technical program for the three-day conference will include keynote talks, refereed full and work-in-progress papers, posters, demo papers and tutorials.

For more information on the conferences conducted by MASCOTS please Click Here.

Large Scale Testing

Many large-scale software systems (e.g., e-commerce websites, telecommunication infrastructure and enterprise systems, etc.) must service hundreds, thousands or even millions of concurrent requests. Many field problems of these systems are due to their inability to scale to field workloads, rather than feature bugs. In addition to conventional functional testing, these systems must be tested with large volumes of concurrent requests (called the load) to ensure the quality of these systems. Large-scale testing includes all different objectives and strategies of testing large-scale software systems using load. Examples of large-scale testing are live upgrade testing, load testing, high availability testing, operational profile testing, performance testing, reliability testing, stability testing and stress testing.

The Large Scale Testing (LT) Workshop focuses on bringing together practitioners and researchers with a interests in Large Systems Benchmarking and Performance Testing.

For more information please Click Here.

OmniTI Surge

Surge started as the brainchild of OmniTI’s employees wanting to bring the best and brightest in Web Operations to their own backyard. Surge has now become the conference on scalability and performance. This conference focuses on topics related to High Performance, Scalability and Reliability. The conference provides an excellent forum where attendees can learn first hand insights and best practices from todayâs brightest minds focused on Web Operations.

For companies experiencing issues associated with growth and scalability of their customer-facing websites, OmniTI is the leading Web Scalability and Performance provider using a cross-disciplinary approach to achieve unparalleled levels of web and database performance. We enable revenue growth, outstanding customer experience, underlying infrastructure reliability and faster time-to-market.

For more details please Click Here. You can see the recordings from the previous conferences here.

PNSQC - Pacific Northwest Software Quality Conference

The Pacific Northwest Software Quality Conference (PNSQC) was first launched in 1982 with a mission to enable knowledge exchange to produce higher quality software. The PNSQC seeks increase the awareness of the importance of software quality with broad interaction and exchange of ideas on both proven and leading edge software quality practices by providing education and opportunities for information exchange within the software community. PNSQC provides a two-day technical program and a one-day workshop during conference on wide range of topics such as Agile, Management, Automation, Mobile and Performance.

PNSQC is aimed at Software product managers, quality professionals, Agilists, managers, contractors, consultants, customers, developer-testers, tester-developers, and maintenance engineers.

For More Details please visit the PNSQC 2014 Website at www.pnsqc.org

Qcon

QCon empowers software development by facilitating the spread of knowledge and innovation in the developer community. A practitioner-driven conference, QCon is designed for technical team leads, architects, engineering directors, and project managers who influence innovation in their teams. QCon highlights the most important development topics driving innovation – things you should be doing now or researching for your next project – presented by the doers in our community.

QCon’s conferences bring practitioners together with attendees who influence innovation in their teams: over half of conferences attendees, for example, have team lead or higher job titles. Additionally, QCons are staged in an intimate environment that promotes high-quality learning, peer-sharing, fun, and inspiration!

QCon’s conferences include tracks focused on Architecture, High Scalability and Systems Performance.

For more information please Click Here.

RAPIDO

The focus of the RAPIDO Workshop is on methods and tools for rapid simulation and performance evaluation in embedded and high performance systems design. Given continuous advances in chip technology, it is to be expected that future-generation processors will integrate numerous units on a single die, including multiple processor cores, multiple levels of (shared/private) caches or memories, and multiple dedicated accelerators, which will be glued together through a network on-chip (NoC).

The design space is huge though:

• How many cores do we need?
• Should we have a homogeneous or a heterogeneous design?
• When dynamic reconfiguration must be performed?
• How many caches/memories do we need?
• How to choose the instruction set(s) for these cores?
• What are the best code optimizations for a given application?
• How to combine the different metrics (e.g. energy, latency and throughput) into a global search space?

All these design questions lead to a huge design space that needs to be explored and which poses a grand challenge to search this space and to deliver an optimal design within the tight time-to-market budget.

To learn more about the RAPIDO workshops please Click Here.

SuperComputing Conference

Established in 1988, the annual SC conference continues to grow steadily in size and impact each year. SC has built a diverse community of participants from across the HPC (High Performance Computing) industry including researchers, scientists, application developers, computing center staff and management, computing industry staff, agency program managers, journalists, and congressional staffers. This diversity is one of the conference’s main strengths, making it a yearly “must attend” forum for stakeholders throughout the technical HPC (High Performance Computing) community.

The technical program is the heart of SC. It has addressed virtually every area of scientific and engineering research, as well as technological development, innovation, and education. Its presentations, tutorials, panels, and discussion forums have.

For more details on the SuperComputing Conference please Click Here.

STPCon (Software Test Professionals Conference)

STPCon or the Software Test Professionals Conference is a conference that brings together professionals in the areas of Functional and Performance Testing with a focus on relevant industry challenges, cutting edge tools, techniques, processes and frameworks.

You can read more about STPCon at Link.

TPCTC (Technology Conference On Performance Evaluation & Benchmarking)

The Transaction Processing Performance Council (TPC) is a non-profit organization established in August 1988. Over the past two decades, the TPC has had a significant impact on the computing industry’s use of industry-standard benchmarks. Vendors use TPC benchmarks to illustrate performance competitiveness for their existing products, and to improve and monitor the performance of their products under development. Many buyers use TPC benchmark results as points of comparison when purchasing new computing systems.

The information technology landscape is evolving at a rapid pace, challenging industry experts and researchers to develop innovative techniques for evaluation, measurement and characterization of complex systems. The TPC remains committed to developing new benchmark standards to keep pace, and one vehicle for achieving this objective is the sponsorship of the Technology Conference on Performance Evaluation and Benchmarking (TPCTC). With this conference series, the TPC encourages researchers and industry experts to present and debate novel ideas and methodologies in performance evaluation and benchmarking.

With the TPC conference, the TPC consortium encourages researchers and industry experts to submit novel ideas and methodologies in performance evaluation, measurement, and characterization.

Topics of interest include, but are not limited to:

• Big Data
• Benchmark Developments
• Appliances
• Cloud Computing
• In-memory databases
• Social media infrastructure
• Business intelligence
• Complex event processing
• Database optimizations
• Green computing
• Disaster tolerance and recovery
• Energy and space efficiency
• Hardware innovations
• Data Integration
• Hybrid workloads
• Virtualization
• Lessons learned in practice using TPC workloads
• Enhancements to TPC workloads

You can read more about TPCTP at Link.

ValueTools Conference on Performance Evaluation Methodologies and Tools

ValueTools is a conference that focuses on methodologies and practices in modeling, performance evaluation and optimization of complex systems, gathering researchers from different communities, such as Computer Science, Networks and Telecommunications, Operations Research, Optimization, Control Theory and Manufacturing.

Topics of focus include:

• Discrete event systems
• Stochastic models
• Simulation techniques
• Game theory
• Queues and network of queues
• Mean field techniques
• Large scale performance analysis
• Control theory
• Machine learning and neural networks
• Performance optimization
• Measurements and benchmarking
• Modeling software tools
• Software Systems

For more information on the conferences conducted by ValueTools please Click Here.

Oreilly Velocity Web Performance & Operations

Building high performance e-commerce websites is tough. Whether you base them on proprietary platforms (ATG, IBM eCommerce, etc.) or Open Source platforms, very rarely do project teams get it right the first time around. Experience tells us that it generally takes a few iterations, multiple code releases and lots of optimization across the application and infrastructure before the web platform has been optimized to deliver the expected performance.

In Oreilly’s own words, “Velocity is much more than a conference; its become the essential training event and source of information for web professionals from companies of all sizes with a focus on Web System Performance & Operations”. Oreilly has designed Velocity to be a place where professionals with interests in Web System Performance & Operations can come together share learning and pick up concepts or ideas related to Web Systems Performance and Continuous/Integrated Operations. As a conference Velocity focuses on the following two main areas:

• Performance of Web Systems
• Agility for Operations i.e. think Devops, think continuous integration.

Through it’s Velocity Conference Oreilly is aims to bring together some of the best and brightest talent in the IT industry with a focus on Web System Performance and Operational Agility. Velocity conferences are being touted as a great place for web ops and performance professionals to come together, learn from their peers, exchange ideas with experts, and share best practices and lessons learned. Velocity Conferences are currently held in Santa Clara, New York, Beijing, and Spain.

For more information visit Oreilly Velocity Web Performance & Operations at – Link.

Workshops On Performance & Reliability (WOPR)

From the WOPR website, “WOPR workshops share experiences in system performance and reliability, and allow people interested in these topics to network with their peers. WOPR is not vendor-centric, consultant-centric, or end-user-centric. We try to create a mix of viewpoints and experiences.

WOPR’s primary focus is on evaluating system performance and reliability. This includes performance measurement, load and stress testing, scalability testing, reliability measurement and evaluation, and system and product certification.

Important but secondary areas of interest include:

• Performance and reliability engineering: system design, network design, database design, self-tuning systems, and self-healing systems.
• System functional testing: test tools and automation frameworks, test planning, data visualization and analysis, test results evaluation, software fault injection, failure analysis, and test lab versus live field testing.
• Operational management: capacity planning, disaster recovery planning, performance tuning, bottleneck identification, troubleshooting, debugging, service level agreements (SLAs) and quality of service (QoS), and managing vendor relationships.
• Management: managing teams who work in the areas above.

The heart of each workshop session is a series of experiential presentations and group discussions, which focus on the chosen topic for the particular workshop. The atmosphere is collaborative, supportive and constructively critical.”

For more information on the workshops conducted by WOPR please Click Here.

WebPerfDays

WebPerfDays follows the “Unconference” format that Brings the Web Performance Community Together. WebPerfDays uses a mixed format that combines formal presentations, ignite sessions, and the open spaces format.

For more information please Click Here.

Copyright Notices – All the images on this page are copyright of their original authors and organizations. Please visit the website of the relevant organizations to learn more about their copyright policies including terms and conditions of use.

Hope you have enjoyed the content on this page. Please send us your thoughts, inputs and comments using the chat interface below.

Trevor Warren is passionate about challenging the status-quo and finding reasons to innovate. Over the past 16 years he has been delivering complex systems, has worked with very large clients across the world and constantly is looking for opportunities to bring about change. Trevor constantly strives to combine his passion for delivering outcomes with his ability to build long lasting professional relationships. You can learn more about the work he does at LinkedIn. You can download a copy of his CV at VisualCV. Visit the Github page for details of the projects he’s been hacking with.