Updated! Updates are shown in quote text like this. Some scores are updated following app updates.

An Apps Experiment

Introduction

This is an experiment I performed out of curiosity, and I have a few big disclaimers at the bottom. Basically, I’ve seen a lot of comments recently about one app or another not displaying something right. Lemmy has been around for a while now and can no longer be considered an experimental platform.

Lemmy and the apps that people use to access the platform have become an important part of people’s lives. Whether you are checking the app weekly or daily, and whether you use it to stay up on the news or to stay connected to your hobby, it’s important that it works. I hope that this helps people to see the extent of the challenge, and encourages developers to improve their apps, too.

How I did it

I wanted to investigate objectively how accurately each app displays text of posts and comments using the standard Lemmy markdown. Markdown is a standard part of the Lemmy platform, but not all apps handle it the same. It is basically what gives text useful formatting.

I used the latest release of each app, but did not include pre-releases. I only included apps that have released an update in the last 6 months, which should include most apps in active development. I was unable to test iOS-exclusive apps, so they are not included either. In all, 16 apps met the inclusion criteria.

I also added Eternity, which is in active development, although it has not had a recent update. I was able to include several iOS apps thanks to testing from @jordanlund@lemmy.world – Thanks, Jordan! This made for 20 apps that were tested.

Each app was rated in 5 categories: Text, Format, Spoilers, Links, and Images. I chose these mostly based on the wonderful Markdown Guide from @marvin@sffa.community, which was posted about a year ago in !meta@sffa.community (here).

I checked whether each app correctly displayed each category, then took the overall average. Each category was weighted equally. Text includes italic, bold, strong, strikethrough, superscript, and subscript. Format includes block quotes, lists, code (block and inline), tables, and dividers. Spoilers includes display of hidden, expandable spoilers. Links includes external links, username links, and community links. Images included embedded images, image references, and inline images.

Thanks to input from others, I also added a test to see if lemmy hyperlinks opened in-app. There was a problem with using the SFFA Community Guide that caused some apps to be essentially penalized twice because there was formatting inside formatting, so I created this TEST POST to more clearly and fairly measure each app.

In each case, I checked whether the display was correct based on the rules for Lemmy Markdown, and consistent with the author’s intent. In cases where the app recognized the tag correctly but did not display it accurately, that was treated as a fail.

Results

Out of a possible perfect 10, 7 apps displayed all markdown correctly:

Alexandrite - 10.0

Connect - 10.0

Jerboa (Official Android client) - 10.0

Photon - 10.0

Quiblr - 10.0

Summit - 10.0

Voyager - 10.0

Arctic - 9.3

Interstellar - 9.1

Lemmuy-UI - 9.0

Thunder - 8.9

Tesseract - 8.6

mlmym - 8.0

Racoon - 7.6

Boost - 7.3

Eternity - 7.0

Lemmios - 6.9

Sync - 6.9

Lemmynade - 6.1

Avelon - 5.7

More details of testing here

Disclaimers

Disclaimers

I Love Lemmy Apps (and their devs)

Lemmy apps devs work very hard, and invest a lot in the platform. Lemmy is better because they are doing the work that they do. Like, a LOT better. Everyone who uses the platform has to access it through one app or another. Apps are the face of the entire platform. Whether an app is a FOSS passion project, underwritten by a grant, or generating income through sales or ads, no one is getting rich by making their app. It is for the benefit of the community.

This is not meant to be a rating of the quality or functionality of any app. An app may have a high rating here but be missing other features that users want, or users may love an app that has a lower rating. This is just about how well apps handle markdown.

This is pretty unscientific

You’ll see my methodology above. I’m not a scientist. There is probably a much better way to do this, and I probably have biases in terms of how I went about it. I think it’s interesting and probably has some valuable information. If you think it’s interesting, let me know. If you think of a better way, PM me and I’d be happy to share what I have so you don’t have to start from scratch.

My only goal is to help the community

I do think that accurately displaying markdown should be a standard expectation of a finished app. I hope that devs use this as an opportunity to shore up the areas that are lagging, and that they have a set of standards to aim for.

I don’t have any Apple things

Sorry. This is just Android and Web review. If someone would like to see how iOS apps are doing, please reach out and I’ll share how we can work together to include them.

  • Plum@lemmy.world
    721·
    4 months ago

    Voyager fangirl here. I have used boost, sync, and jerboa, and voyager won pretty quickly.

    • 667@lemmy.radioEnglish
      36·
      4 months ago

      Having come from Apollo, the dev(s?) really focused on parity and it made the changeover simple.

      • Plum@lemmy.world
        10·
        4 months ago

        I used reddit is fun exclusively. Once I figured out compact post size, I was sold. It has every functionality and a great UI and it hardly ever barfs. And it has a Dracula theme for dark mode. Brilliant. Seamless.

    • flying_sheep@lemmy.ml
      5·
      3 months ago

      Similar progression here. The only thing I don’t like in Voyager is the iOS-isms.

      But that’s minor, I got accustomed to the weird icons and so on pretty quickly.

      Excellent app, must be absolutely perfect on iOS!

      • curbstickle@lemmy.dbzer0.com
        2·
        3 months ago

        Yup.

        Visually I like Thunder quite a bit, but it’s still fairly early in their development so it doesn’t have feature parity yet.

        Voyager, while stylistically not my favorite, is the most performant and usable out of any of the apps I’ve tried. It’s what I use most often (though I still have a ton of Lemmy apps installed and occasionally switch around on a new release to try them out).

        Relay and Sync were my preferred Reddit apps, Relay probably got the longest use for me, though Sync was the last I used regularly before the API shenanigans.

        • micahmo@programming.dev
          1·
          3 months ago

          Visually I like Thunder quite a bit, but it’s still fairly early in their development so it doesn’t have feature parity yet.

          Just out of curiosity (no need to answer if you don’t want to) are there any major features missing from Thunder that you’d like to see? I’m not trying to sway you back or anything (it’s not like we have any incentive for getting more people to use Thunder); I’m just looking for ideas to make it as good as possible!

          • curbstickle@lemmy.dbzer0.com
            2·
            3 months ago

            It’s not like I don’t have thunder installed (I’ve actually got it on a few devices), just I don’t open as often. Main reason historically has been content display - usually less link previews, and more often attached local media. Haven’t rechecked in a bit, but I think it’s been a couple months since I had seen an update in fdroid, which is usually my signal to check out the latest.

            Feature-wise? Multis would be amazing. Especially with federation where multiple servers have related communities, being able to group what I want to see is awesome. Raccoon just added a (partial, new, maybe problematic) version. Ideally this should be server side, but I don’t expect it’s a high priority item.

            In terms of what I can recall off the top:

            • Images would get squished in full height mode
            • Clear read posts is a super useful one in Voyager, a nice-to-have for thunder
            • Had an issue with messages, unable to reply from inbox maybe? Can’t remember exactly what it was.

            Regardless, when the updates roll out to the fdroid repos, I’ll be updating and checking it out. I’ve got logins at multiple instances, for different tasks (programming, memes, being hopeful that we won’t have a criminal as president, etc), some clients can be better than others at commenting or browsing or just the way markup looks. So it’s less to me about “best” and more “what I feel like using right now” if that makes sense. Which could be the same thing, but isnt always.

            Always happy to try out a new rev though!

            • micahmo@programming.dev
              2·
              3 months ago

              Hey, thanks so much for the response! It’s incredibly useful to hear real feedback like that, and I appreciate the time you took to write it all up.

              I think it’s been a couple months since I had seen an update in fdroid, which is usually my signal to check out the latest.

              Fair enough, this latest release cycle has been a little slower, but there is a new update coming soon. We’re also working on putting pre-releases on F-Droid in a different channel for those who want to be on the bleeding edge (the pre-releases come out much more often).

              Multis would be amazing.

              I assume you mean multi-communities, where you can view a feed of posts from multiple communities at a time? If that’s what you mean, does this GitHub issue cover your ask?

              https://github.com/thunder-app/thunder/issues/13

              Images would get squished in full height mode

              This one has been hard for the devs to reproduce, but you’re not the only seeing it. If you want to follow progress on fixes, check out this issue.

              https://github.com/thunder-app/thunder/issues/1448

              Clear read posts is a super useful one in Voyager, a nice-to-have for thunder

              This one we do have! It’s available via the Floating Action Button. By default you can long-press or swipe up on the FAB to see additional options, including “Dismiss Read”. But the FAB is also super customizable, so you can make that the default action or the long-press action if you want.

              Had an issue with messages, unable to reply from inbox maybe? Can’t remember exactly what it was.

              As part of the upcoming release, the inbox has been redesigned, so hopefully replying should work now!

              Anyway, no need to respond, just wanted to follow up and say thank you again for the time and the thoughts.

              • curbstickle@lemmy.dbzer0.com
                2·
                3 months ago

                Excited to check the new version out!

                I’ll take a look at the GH issues in a bit (super fun work day full of mariadb, disk resizing, and an unfortunately large amount of ec2 efforts…)