Changes between Initial Version and Version 1 of Develop/Drafts/LayersAndAnimation

Oct 8, 2016, 5:10:45 PM (7 years ago)



  • Develop/Drafts/LayersAndAnimation

    v1 v1  
     1= Introduction =
     3Two of the big features we plan to add in GrafX2, layers and animation, need a
     4rewrite of the management of pages in the program. This page gather the ideas
     5and possible designs and is open for discussion.
     7= User interface overview =
     8== Animation ==
     9The idea is to provide a full animation package allowing to create full
     10video sequences. This would be based on effects applied to layers : rotation,
     11distorsion, movement and zoom. We also provide regular animation with simple
     12frame switching and different drawings on each frame, and palette cycling.
     13It's not so easy to combine all this in one coherent interface.
     15The timeline is the main thing we create when we open a file. It is possible to
     16render a specific frame to the screen. Rendering all the frame according to
     17their timestamp will play the animation. Rendering only one frame allows to
     18preview it. It is possible to switch between frame preview and layer edition.
     19It is not possible to paint on a frame, because the layers can be rotated, zoomed,
     20distorted. You have to select a layer and show it in 1:1 pixel mode to edit it.
     21In this mode the other layers are no longer linked to the selected one, so you
     22edit only one layer, on a black background. We could use the splitscreen system of
     23the zoomed view to display both the frame rendering (as the normal view) and the
     24edited layer (as the zoomed view).
     26An animation is a set of frames arranged in a timeline. Each fram can have a
     27different palette, so it is possible to use them to do simple palette cycling
     28animations. They also hold a list of layers. Each layer is composed of two
     29parts: picture data and distorsion information. Picture data is the real set of
     30pixels. The same picture data can be used in multiple layers, either in multiple
     31frames (this is what happen when you do palette cycling: the same pixels are
     32dispayed over and over, with a different palette). The distorsion is the
     33position, rotation, stretch and linear distorsion of the layer. It can be
     34handled pretty much like the current "brush effects" screen. So you can have a
     35part of your picture scrolling over the screen while rotating, if you want.
     37Different palette for each animation step: I don't like it very much, as it's
     38quite painful to keep the palette consistent over the different frames, and I
     39really doubt it's well-supported in GIF format (the specs says yes, but I've
     40seen how web browser and Windows image editors don't support the specs for
     41animation). Palette animation is something you generally do without changing
     42pixel data... I suggest we implement it like Deluxe Paint, completely
     43independant from image animation. This will not allow to do things like
     44fade-in/fade-out in a moving animation. I think we can have a "palette cycling"
     45interface in the palette menu, and a more powerful palette management in animation
     46mode, both using the same backend.
     48I saw some quite advanced features in picfmts.doc (don't remember which format),
     49in this one it was possible to have each cycling step stay on screen for a
     50different amount of time, and the cycling going once, in a loop or in "pingpong"
     51mode. So I tought it made sense to use the same timeline system as the other
     52animation features. This is practical for us on the code size, not meaning we
     53enforce people to use both features in their works. If the cycling is independat
     54from the animation, it means it is not possible to change the palette in an
     55animation in any easy way, and it also means we go for a very simple cycling
     56scheme with colors looping in an endless loop and staying each for the same time
     57on screen. I find that a bit sad. Remeber we are trying to be the ultimate 256
     58color painting program ;)
     60There is a thing that i'm not sure: should the spare page be also an animated
     61timeline ? or should just it be a still frame with fixed layers ? or just a
     62single page ?
     65There are two ways to use layers :
     66  * In a still picture, the photoshop way. (this is what a layer usually is)
     67  * In an animation, which is what I was thinking of when writing this page. In this case a layer is used as a sprite and can be stretched, rotated and distorted. But to draw on it, it is important to have an undistorted view. When working on an animation, we want to display only the active layer. When working on a still picture, we want to display all of them. Maybe the two modes are too different and incompatible, or maybe we can find a way to make them work together.
     69Proposition of UI for the 'layer' buttons, inspired by Artgem and completed with
     70experiments with a tile editor which supported layers :
     71  * Each "layer" button shows one of 3 possible states : Hidden, Visible, Visible+Current (where you draw)
     72  * Only one layer is Current at a given time.
     73  * Clicking a non-current layer with RMB: toggles it visible on/off
     74  * Clicking a non-current layer with LMB: it becomes (visible and) Current.
     75  * Clicking the current layer with LMB:
     76    * Not effect. Maybe we can use it to initiate drag-n-drop operation: dropping on another layer button will cause a translation of the current layer to target position.
     77  * Clicking the current layer with RMB:
     78    * If it's the only visible layer, all layers become visible,
     79    * Else, it makes all other layers invisible.
     82The number of layers is limited anyway, because we can't put an infinite number
     83of keyboard shortcuts... "Switch to layer 1", "Toggle layer 4", etc. One very
     84handy keyboard shortcut can be to "Toggle current layer", because it will switch
     85from "all layers" to "only the current layer", just like when you right-click
     86the current layer.
     88Further experiments with UI:
     92The vertical layouts are the most natural, since the "bottom" layer is the
     93background, and the one over it are piled on top of each other. But the size is
     94VERY limited as it makes tiny buttons, 8 layers is really a maximum. The buttons
     95are still bigger than those in ArtGem though, even when the menus are not zoomed.
     96Some of the vertical scemes remove the separation between the 4 parts of the menu
     97(tools, effects, settings, palette), which does not seem a good idea. The
     98separation bars could be made thinner if we need some space, but I prefer we keep
     99them. I think the layers would be part of the settings+management part, along with
     102Horizontal layouts allow more layers, but make the menu bigger (when layer row
     103is displayed). The horizontal layerbar provide a button to show or hide it. This
     104is a good idea, however the button to do the same with the menu is in the status
     105bar. It'd feel strange if hiding the statusbar also hide the menu but not the
     106layerbar. I like this system as we also will need an animation bar and maybe
     107some others. The layer buttons are not aligned with the buttons in the menu, but
     108this can't be solved. Not really a big problem. The "Layer" button in the bar
     109should use the same font as the KILL and PAL buttons.
     111Ilkke suggested to replace the Hide button with one to switch between multiple
     112toolbars/modes. The default position would be the current one, left click would
     113switch from/to animation mode (timeline, keyframes management, playing speed,
     114looping mode (no loop/loop/pingpong), play, stop buttons, etc.). RMB would
     115switch the menubar off and stay in the current mode (useful in animation mode to
     116watch your work in fullscreen). Maybe the layer part could go either on a third
     117bar or with the animation. However this is unpractical when drawing a still
     118picture if you want to switch quickly between the layers... Combining this with
     119the "stackable toolbars" idea seems nice.
     121Possible alternative: button opens a popup menu to choose which toolbar(s) are visible:
     125Another solution is to keep the menu as it is and use the space where we have the
     126palette as an area where you can display palette or layers or animation. The
     127settings/load/save part of the menu could be moved there as well (you don't use
     128it often).
     130I'm not sure it's easy to remember which number is attached to which layer.
     131Maybe a preview system (like the one in the load/save) screen could be useful
     132for that. This could also allow easier palette sorting, maybe (with a list of
     133layers and some move up/down/top/bottom buttons). Button dragging mean we have
     134to write special code for draggable buttons. But could be nice for quickly
     135moving things around.
     137= iLKke's mockups =
     140We spent some time with Ilija trying to tune up this. The horizontal strip seems more appropriate.
     142*Note on mockup:*
     143In the horizontal strip, some buttons have RMB functions: add+rmb=clone; merge+rmb=flatten (merge all)
     145= iLKke's ideas & thoughts =
     146These are provided here so you can all read them. The text can then be integrated with the rest in a more tidy manner. Excuse the formatting.
     148LAYER MENU
     152 pros:
     154 * uses spare space
     155 * vertical layer display makes sense
     157 cons:
     159 * tiny buttons(1)
     160 * clutters the rest of the GUI if separators are removed(2)
     164 pros:
     166 * more space for buttons and overview, also for basic functions
     167 * would have to be done for animation anyhow
     169 cons:
     171 * takes up more screen space(3)
     173(1)tiny buttons can be countered by having an adaptive button size (as on mockup)
     175(2)use smaller separators (as on mockup)
     177(3)brilliance proved this to not be a real problem as long as you can hide/show all menus via keyboard
     179All in all, horizontal seems to work better.
     181THUMBNAIL PREVIEW (for horizontal menu only)
     183 cons:
     184 * takes up a lot of screen space
     185 * might prove unreadable most of the time, even at relatively large thumbnail sizes
     187 possible solutions:
     188 * a set of assignable icons/colors for layer buttons instead of thumbnails(1)
     189 * thumbnail preview only available in special layer popup/arranger(2)
     191(1) external data that would need to be saved somehow, also, active/hidden layers would have to be denoted using other means than color.
     193(2) will make a quick mockup of it at some point, but this is not ideal since it would be optimal to have all or most parts of layer GUI in one horizontal strip
     197-move layer up/down - for arranging layers (already in mockup)
     199-flash current layer - for quickly identifying layer contents. can be a key shortcut, or optionally when you select a layer it would flash briefly (toggable in prefs). flashing would be changing the colors of the layer contents for a brief period of time, perhaps to black then white. perhaps someone has a less aggressive idea.
     200 UPDATE: optionally the layer could shake very briefly (move by a pixel vertically then back) or blink (go off/on) instead of the epilepsy-inducing flashing.
     202-load/save individual layers - quite useful. perhaps add optional merging of palettes, etc
     204Those last couple of features that haven't been included in the strip can be available by clicking on the 'layers' button on the left of the strip. Or included in the strip, somehow.
     206= Layer menu =
     208In any case, a "layer menu" button is not necessary, it can be replaced by double-clicking the active layer.
     210The "layer menu" can propose:
     211    * Set transparent color
     212    * Add empty layer
     213    * Clone current layer
     214    * Delete current layer
     215    * Flatten all
     218 Functions that have dedicated buttons should be removed from the layer menu (all of the above?). Instead, two important functions could take their place:
     219    * Load layer
     220    * Save layer
     222The existing "Copy to spare" menu will behave differently if the spare is a multi-layer image:
     223    * The copy of pixels (choices 1,2 and 4) will copy current layer to (replace) spare's current layer, without changing the spare's dimensions.
     224    * "Palette and Remap" will remap all the layers of the spare.
     225This will be the method to import an image as a new layer: Add new layer, Swap to spare, load image, 'Copy to spare' / 'Pixels'.
     227I would really prefer using only one color index, of user's choice, as
     228transparent within the image. It will be transparent for us for all layers
     229except the last visible layer, where the color shows. I know it's traditional
     230for windows program to show a checkered pattern, but it's going to be slow for
     231us and it's equally traditional for sprite artists to draw on "magic pink" or
     232any color of the right luminance as the expected background that is clearly not
     233in their palette. The only drawback is all the layers are then 255 colors only.
     234Independantly from this, the user can still choose a "transparent color for GIF
     235or PNG".
     237Actually I was thinking of something more like the "Stencil" system we have,
     238but it would make the selected colors transparent, instead of not allowing you
     239to draw on them. But maybe it's overkill, if I want to make part of a drawing
     240appear and disappear in an animation I'll just put these part on a separate
     241layer, and make it invisible as a whole. So let's go for a single transparent
     244===Shared layers===
     245Optionally, with animation and layers, we could implement Shared layers.
     247A shared layer is a layer which is present in more than one step of the animation. A classic use is to make your background layer shared, so it is present in every step of animation, and drawing on it will affect all the steps at a time.
     249This system doesn't allow a layer that scrolls/pans as an animation... It would require a full recomputing of all frames every time you draw something, it's too slow to do it dynamically.
     251In terms of UI, we have to think a bit. The system can handle a lot of things, but it's not easy to make it clear and user-friendly.
     253For the implementation, it's rather easy because we'll already have reference-counting in place to handle backup steps. We only have to be careful when saving/loading, because it would automatically separate the layers on saving.
     255= Architecture overview =
     257Both layers and animations are part of the code refactoring. Animation things
     258will be made available in the gui later.
     262At the top you see the current system.
     263At the bottom it's the new one.
     265= Notes =
     266We introduce three new structures: T_Timeline, T_Frame, and T_Layer, and we do
     267some changes in the T_Page and T_List_of_page.
     270Its a linked list of T_Frames, sorted by timestamp.
     271The T_Timeline holds the comment, filename, format and directory of the current
     272edited file. It is possible to have only one T_Frame, one T_Layer and one T_Page
     273to handle usual still picture files.
     276This holds the data for a frame: palette, timestamp, and a linked list of
     277layers. The palette is stored here because it needs to be global to the frame
     278(we allow 256 colors on screen at once, no more) and because this allows to
     279display the same page with different palettes for palette cycling.
     281Of course the list of layers is ordered by Z-Order, from bottom to top, this
     282allow easy rendering of the frame by simply rendering the layers in the order
     283they come in the list.
     286This is the struct defining a layer. It holds a pointer to the picture data (this
     287picture data can be used in multiple layers, in the same frame or in different
     288successive frames). It also holds the transparent color number (this one can
     289also be different for each occurence of the picture data) (maybe there should be
     290multiple transparent colors ?), and finally, the distorsion is handled there
     291simply by storing the screen pixel coordinates of the 4 corners of the layer.
     292This allow to render any kind of distorsion using only the linear_distort
     293function we have for brushes. However, it's pretty hard to generate this data by
     294hand to make a proper rotation, so we will need a powerful way of generating
     295them. We may want to store the parameters used for this generator (lets say,
     296translation vector, rotation speed, zoom ratio, and simple offset for
     297hand-distorsion of each of the 4 corners), allow the user to tweak them at any
     298time to change his animation, then interpolate them to generate our distorsion
     299points. This would then be some kind of keyframe you place at main points of the
     300animation, while the other frames are interpolated from them.
     303A T_Page holds a bunch of pixels with the width and height. It can be packed
     304using some RLE and delta system to avoid wasting memory. However, I think all
     305pages used in the animation should be depacked before playing. This can be done
     306at the same time we generate interpolated frames from the keyframes, when you
     307click the "render and play" button. The packing is particularly interesting for
     308backup pages that, most of the time, are not used (particularly the old ones).
     309Actually, the delta only make sense in the backup list because this animation
     310system does not provide a way to have ordered pixel data to make an hand-drawn
     311animation. If you do that, it's just a set of frames with different pages of the
     312same size, the same palette, and the same distorsion everywhere. But the T_Pages
     313don't know they are ordered in an animation. They can even be used in completely
     314unrelated places and without animation (or in reverse order) at another place.
     316As a T_Page can be used in multiple layers, i think it makes sense to add some
     317simple reference counting mechanism. Each time an object get a pointer to a
     318page, it increments a counter. When the object is deleted or release its pointer
     319in any way, it decrements the counter. When the counter reaches 0 the page could
     320be deleted immediately. Or it could be kept in packed form in some kind of
     321pagelist, allowing to use it later in another layer. This is also used for
     322backups: when you backup a picture you only save the modified layer and copy the
     323references for all the others.
     326This is used in the current backup system. I'm not sure how we should handle
     327undo/redo with the animation. We can have a separate undo list for each T_Page,
     328or for each T_Frame (allownig to undo any change including palette, distorsion).
     330If the backup list is a list of frame, then it is possible to undo frame
     331deletion, because we can push the frame back in the main list thanks to its
     334The way I see it, most editing actions modify only one layer of the current
     335frame. Some actions DO modify several or all frames, like for example a resize.
     336With a smart system of reference counting, we can point to the same pixel data
     337(T_Page) for the different "editing steps" of layers that couldn't be affected
     338by the operation. I find this idea critically important for performances,
     339because on each action (brush stroke) we need to backup for the undo/redo
     340system, and not having to duplicate a dozen of frames is going to be a
     343= Rendering, display to screen =
     344For the drawing, actually, we will have to have 3 things in memory:
     345  * The current layer
     346  * The rendering of all the other layers (both top and below)
     347  * A mask saying for each pixel in which layer it actually is. By testing this value against the current selected layer we can tell if the pixel is modifiable in a visible way or not. Storing the layer numbers in this plane allow us to keep it when the user go to another layer.
     349Remember that almost all effects need to read from the "backup page". For each
     350tool there is always the question if it should read from the last backup of
     351"only current layer (brush grab)", or "current layer and all visible layers
     352below it" (like smear, you smear under the top layers) or "all visible layers".
     354This is the most difficult thing to handle. Having it behave differently for
     355each tool seem uneasy to learn, having it as an option is too much buttons
     356(feeture creep as they say), and I think a single setting for all of them will
     357not make sense. One solution could be to take from all visible layers, and let
     358the user swich off the ones he don't want to use. But this means a lot of
     359playing with layers. Maybe a feature to store the current visibility of layer
     360and then restore it afterwards can do the trick. But, it's still not a perfect
     363=File format=
     364We will have to design our own format to store all of this in a convenient form.
     365PKM can be extended for that using more/different chunks, not necessarily
     366keeping compatibility and name, but using the same chunk system. We will want to
     367import palette cycling from neochrome, degas elite, and IFF format, and to
     368save/load gif animations. Maybe we can find infos on what other animation
     369package such as Autodesk Animator Pro had (features, file format, ...)
     371== Use extensibility of PNG and GIF ==
     372PNG and GIF allow us to add custom chunks of data that will (should?) be ignored
     373by viewer programs. The files will be quite big, because we basically store all
     374the individual layers in addition to the "visible" flattened image.
     376== Rely on standard GIF behavior ==
     377For layers, one early/easier method is to use GIF to store layers as individual
     378frames with a delay of 0 and no repetition. In theory it's the ideal file format
     379as it has no size overhead, unfortunately many GIF viewers impose a minimum
     380delay of 1/10s or 1/20s (Firefox, IE), and some are unable to display a framed
     381GIF without repeating it in a loop. Still we can use it as an internal format.
     383Here's an example of such image:
     387Technically it's a 8-frame GIF. Frames are background, characters, GUI, Title, and 4 more unused (empty ones).
     388Dispose method is "do not dispose", and time of each frame is 0.05s. The NETSCAPE2.0 extension that requires looping
     389is NOT present, so it shouldn't loop.
     391This loads as a layered image in GIMP, so I guess using this format will make everyone happy. However, it does not show properly in geeqie (linux image viewer), only the first frame is there because they don't support animated gif. Opera 10 seems to show it fine.
     393== VGAPaint example ==
     395I launched vgapaint to see what it had. There are no layers. The animation is
     396bitmap-frame based, and the palette is handled a bit like what I did for pixel
     397data, ie a single palette can be shared between multiple frames, but you can
     398also decide to use another one. There is nothing much more to say. The function
     399to import a set of numbered PCX from hard drive looks interesting (load
     400ANIM0001.PCX as first frame, ANIM0002.PCX as second, and so on). There is sound
     401support (well... I think we don't need that), AVI import/export, and a custom
     402format, .VAN. I've uploaded a wiki page called [VGAPaintMenus] if you want to
     403have a look at it (just so it's linked from somewhere :p).
     405I had a look at the VAN file format. It's quite primitive actually. There is a
     406magic number, a small header with number of frames, framerate, width, height,
     407and palette, then each frame also starts with a magic number, a different one
     408depending on if there is a palette or not; there is the palette and the pixel
     409data ,either as RAW or RLE-encoded. So it's some kind of chunk based format but
     410does not sound as clean as our pkm (no size indication at begin of chunks, so no
     411way to skip a chunk). I also find VGAPaint's code less easy to read than ours...
     413= Use of animations =
     414Of course playing with animations is nice, but if they can't go out of the
     415program, it's of little interest. Gif export is already a nice solution, but the
     416ability to play your own format in another program directly would be really nice.
     418The idea is to provide a piece of code to render and play the animation in any
     419program using SDL. In this context, another program could ask us to render a
     420frame to an SDL screen, either with some content already in it, or just filled
     421with magic pink for blitting later. The frame render should not touch the unused
     422pixels, it should not fill the screen in black. It'd be really nice to be able
     423to control the engine and use it to build simple games (let it load all the
     424layers, but build the timeline in realtime depending on what the player does).
     425But maybe i'm dreaming too much :). Rendering over a 24bit screen could also be
     426a nice feature, in order to have a 24bit background and then simple layers in
     427256colors over it. Or a 3d game with some animation to make the ingame display.
     428But, let's start with simple things :)