News
LET’S DRAW!: New Deep Learning Technique for Realistic Caricature Art
Press Release
LET’S DRAW!: NEW DEEP LEARNING TECHNIQUE FOR REALISTIC CARICATURE ART
Data-driven method leverages deep learning AI for photo-to-caricature translation
TOKYO, Japan, 16 October 2018 – Caricature portrait drawing is a distinct art form where artists sketch a person’s face in an exaggerated manner, most times to elicit humor. Automating this technique poses challenges due to the amount of intricate details and shapes involved and level of professional skills it takes to transform a person artistically from their real-life selves to a creatively exaggerated one.
A team of computer scientists from City University of Hong Kong and Microsoft, have developed an innovative deep learning-based approach to automatically generate the caricature of a given portrait, and to enable users to do so efficiently and realistically.
“Compared to traditional graphics-based methods which define hand-crafted rules, our novel approach leverages big data and machine learning to synthesize caricatures from thousands of examples drawn by professional artists,” says Kaidi Cao, lead author, who is currently a graduate student in computer science at Stanford University but conducted the work during his internship at Microsoft. “While existing style transfer methods have focused mainly on appearance style, our technique achieves both geometric exaggeration and appearance stylization involved in caricature drawing.”
The method enables users to automate caricatures of portraits, and can be applied to tasks such as creating caricatured avatars for social media, and designing cartoon characters. The technique also has potential applications in marketing, advertising and journalism.
Cao collaborated on the research with Jing Liao of City University of Hong Kong and Lu Yuan of Microsoft, and the three plans to present their work at SIGGRAPH Asia 2018 in Tokyo from 4 December to 7 December. The annual conference features the most respected technical and creative members in the field of computer graphics and interactive techniques, and showcases leading edge research in science, art, gaming and animation, among other sectors.
In this work, the researchers turned to a well-known technique in machine learning, Generative Adversarial Network (GAN), for unpaired photo-to-caricature translation to generate caricatures that preserve the identity of the portrait. Called “CariGANs”, the computational framework precisely models geometric exaggeration in photos (shapes of faces, specific angles) and appearance stylization (look, feel, pencil strokes, shadowing) via two algorithms the researchers have labeled, CariGeoGAN and CariStyGAN.
CariGeoGAN only models the geometry-to-geometry mapping from face photos to caricatures and CariStyGAN transfers the style appearance from caricatures to face photos without any deformation to the geometry of the original image. The two networks are separately trained for each task so that the learning procedure is more robust, notes the researchers. The CariGANs framework enables users to control the exaggeration degree in geometric and appearance style by dragging slides or giving an example caricature.
Cao and collaborators conducted perceptual studies to evaluate their framework’s ability to generate caricatures of portraits that are easily recognizable and not overly distorted in shape and appearance style. For example, one study assessed how well the identity of an image is preserved using the CariGANs method in comparison to existing methods for translating caricature art. They demonstrated, through several examples, that existing methods resulted in unrecognizable caricature translation. Study participants found it too difficult to match the resulting caricatures with the original subjects because the end results were far too exaggerated or unclear. The researchers’ method successfully generated clearer, more accurate caricature depictions of portrait photos, as if they were hand drawn by a professional artist.
Currently, the focus of this work has centered on caricatures of people, primarily headshots or portraits. In future work, the researchers intend to explore beyond facial caricature generation into full body or more complex scenes. They are also interested in designing improved human-computer interaction (HCI) systems that would give users more freedom and user control on the machine learning-generated results.
For the authors’ paper, titled “CariGANs: Unpaired Photo-to-Caricature Translation”, and supplementary material, visit the team’s project page.
To register for SIGGRAPH Asia, and for more details of the event, visit http://sa2018.siggraph.org. For more information on the Technical Papers Program, click here.
ACM SIGGRAPH Bylaw Changes Passed
The proposed bylaw changes by the ACM SIGGRAPH Executive Committee (EC) have been approved by the ACM EC when the member voting failed to meet the ten percent rule required for all bylaw changes. These changes will start in the voting taking place in next summer's elections.
The biggest bylaw change is all elected positions will become director positions rather than electing a person to a specific position such as the President, Vice-President, and Treasurer as we have done in the past. The EC will select new officers from within the EC to serve one-year terms. The officers will be the Chair, Chair-Elect, Treasurer, and Treasurer-Elect. The Chair-Elect will become the Chair and the Treasurer-Elect will become the Treasurer the following year.
As some of you know, choosing to volunteer, particularly for these key leadership roles, is a huge time commitment. Many of us are at junctures in our careers where we may not be able to volunteer for one of these larger roles due to time constraints; this new structure within the EC would share the work more broadly across the directors of the EC and lessen the burden on a specific position. It is our hope in adopting this new structure our nominating committee will have a larger, more diverse pool of candidates who would consider taking on a leadership position within the EC.
The SIGGRAPH EC worked with experts in nonprofit governance to put forth modifications around the organization’s structure and processes of voting/elected positions. Internal elections are becoming standard practice for nonprofit, volunteer-run boards, as are one-year terms for the elected officers.
The second important change revolved around creating a structure that accurately reflects our diverse community and will enable us to be more representative of our membership. Instead of being presented with a total of N+1 candidates for N positions as is specified in the previous version of the Bylaws, voters will be presented with at least two candidates for each specific position. This will enable the ACM SIGGRAPH Nomination committee to “bucket” candidates into positions to ensure all open director roles aren’t filled, for instance, with members who are from academia or that the EC doesn’t have only representation from North America. With this modification to the Bylaws, the nominating committee is able to achieve increased diversity in skillset, area of expertise, and geography on the EC.
Lastly, the third major change to the Bylaws is the EC will be allowed to appoint three voting members to its rank. Currently, the both Chairs of the Conference Advisory Groups for the North America and Asia, for instance, serves in an ex officio, non-voting capacity. As we are all aware, both SIGGRAPH conferences are our group’s marquis outward-facing programs, and we feel it is important to have at least one of these individuals in a voting position. This change will also allow the EC to increase representation and expertise from core constituencies as needed to complement the make-up of the elected EC.
The EC worked on these modifications for the past two years to achieve a more modern, efficient, flexible, and diverse volunteer-run operation. These Bylaw changes will allow us to be more representative of the ACM SIGGRAPH membership, and in turn, help us create a strategic direction and programs that will better benefit the entire community.
Nomination for SACAG Chair
Nomination for SACAG Chair
The current SIGGRAPH Asia Conference Advisory Group (SACAG) Chair, YT Lee, will be stepping down at the end of 2018. The nomination for the new Chair is now open. Here are the details:
1. Anyone who knows a suitable candidate can make a nomination.
2. Please provide the justification for your nomination, highlighting your candidate’s services for the SIGGRAPH organization, particularly for SIGGRAPH Asia.
3. The nominee must consent to be nominated. Please state that in your justification.
4. The closing date: 31 Oct, 2018, 23:59 Eastern Time.
5. Please send your nomination to: sacagnomination@siggraph.org.
6. The SACAG will select the candidate by voting, subject to approval by the ACM SIGGRAPH executive committee.
7. The result will be announced at the SIGGRAPH Asia Conference in Tokyo.
Volunteer Opportunity: ACM SIGGRAPH External Relations Committee Chair
Volunteer Opportunity: the ACM SIGGRAPH Executive Committee is Recruiting for an External Relations Committee Chair
The ACM SIGGRAPH Executive Committee is recruiting for a Chair of their External Relations Committee. The chair and their committee will establish and oversee ACM SIGGRAPH's agreements with organizations and conferences worldwide; manage relationships with external professional societies, organizations, and companies; and work with the ACM Executive Committee to identify and pursue relationships with new organizations and companies in alignment with the current strategic plan.
The External Relations Committee will also work with the Specialized Conference Committee to strengthen existing ties and identify new venues, emerging themes or potential relationships with other conferences and organizations to broaden the scope of SIGGRAPH.
The Chair of the External Relations Committee will assemble a small committee of 2-5 individuals interested in strengthening the relationships between the SIGGRAPH organizations and other organizations and companies. The committee members should be selected from different sub-fields of graphics so that they can work with organizations and companies related to their area of expertise such as displays, design, and robotics. They should also be geographically diverse because many of our agreements are with organizations outside the U.S.
Some of the current partnership agreements that have been cultivated by former external relations chairs include:
- Computer Graphics Arts Society (CG-ARTS)
- Digital Content Association of Japan (DCAJ)
- Eurographics
- Laval Virtual
- Spark Computer Graphics Society
- VFX Rio
- Unity
- Nvidia
The Executive Committee would like to broaden out the areas covered by the organizations and companies that SIGGRAPH has relationships with to include areas that are synergistic with computer graphics and interactive techniques and aligned with the organizational mission of “Enabling Everyone to tell their Stories.”
We estimate that this role will require about two hours/week.
If you are interested in this position, please contact: Rebecca Strzelec